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November  4,  1993 

MEMORANDUM  FOR  DIRECTOR,  DEFENSE  RESEARCH  AND  ENGINEERING 

ASSISTANT  SECRETARY  OF  DEFENSE  (COMMAND,  CONTROL, 
COMMUNICATIONS  AND  INTELLIGENCE) 

ASSISTANT  SECRETARY  OF  THE  NAVY  (FINANCIAL 
MANAGEMENT) 

ASSISTANT  SECRETARY  OF  THE  AIR  FORCE  (FINANCIAL 
MANAGEMENT  AND  COMPTROLLER) 

DIRECTOR,  ADVANCED  RESEARCH  PROJECTS  AGENCY 
DEPUTY  UNDER  SECRETARY  OF  DEFENSE  (LOGISTICS) 
AUDITOR  GENERAL,  DEPARTMENT  OF  THE  ARMY 

SUBJECT;  Audit  Report  on  Sole-Source  Justifications  for 
DoD-Sponsored  Federally  Funded  Research  and 
Development  Centers  (Report  No.  94-012) 

We  are  providing  this  final  report  for  your  information  and 
use.  The  audit  was  requested  by  the  Administrator,  Office  of 
Federal  Procurement  Policy,' and  House  Report  No.  102-95, 
"Department  of  Defense  Appropriations  Bill,  1992."  This  is  the 
second  in  a  series  of  reports  on  DoD-sponsored  federally  funded 
research  and  development  centers.  Management  comments  to  a  draft 
of  this  report  were  considered  in  preparing  the  final  report. 

Based  on  management  comments,  we  revised,  added,  and  deleted 
recommendations.  DoD  Directive  7650.3  requires  that  all 
recommendations  be  resolved  promptly.  We  request  the  Director, 
Defense  Research  and  Engineering,  to  provide  comments  on 
Recommendation  1.  and  the  Navy  to  provide  comments  on 
Recommendation  3.  and  the  monetary  benefits  by  January  4,  1994. 

We  appreciate  the  courtesies  extended  to  the  audit  staff. 

If  you  have  any  questions  on  this  final  report,  please  contact 
Mr.  Garold  E.  Stephenson,  Program  Director,  at  (703)  692-3179 
(DSN  222-3179)  or  Mr.  John  M.  Gregor,  Project  Manager,  at 
(703)  692-3205  (DSN  222-3205).  The  distribution  of  the  report  is 
listed  in  Appendix  T. 


Robert  ff.  Lieberman 
Assistant  Inspector  General 
for  Auditing 


Office  of  the  Inspector  General,  DoD 

Report  No.  94-012  November  4,  1993 
(Project  No.  lCH-5012.02) 

SOLE-SOURCE  JOSTIPICATIONS  FOR  DOD-SPONfiORED 
fEnBRaLLY  EDNDED  RESEARCH  AND  DEVELOPMENT  CENTERS 

BYECUTIVE  SUMMARY 

Introduction.  On  March  12,  1991,  the  Administrator,  Office  of 
Federal  Procurement  Policy  (OFPP) ,  requested  that  selected 
Inspectors  General  determine  whether  sponsoring  agencies  were 
complying  with  OFPP  Letter  84-1,  "Federally  Funded  Research  and 
Development  Centers  (FFRDCs)."  On  September  16,  1992,  the 
Administrator,  OFPP,  further  requested  that  the  Inspectors 
General  assess  the  adequacy  of  the  sole— source  renegotiation 
justifications  for  sponsored  FFRDCs.  House  Report  No.  102-95, 
"Department  of  Defense  Appropriations  Bill,  1992,"  likewise 
requested  the  Inspector  General,  DoD,  to  review  the  use  of 
FFRDCs . 

OFPP  Letter  84-1,  as  implemented  by  Federal  Acquisition 
Regulation  35.017,  "Federally  Funded  Research  and  Development 
Centers,"  provides  Government -wide  policy  for  the  establishment, 
use,  periodic  review,  and  termination  of  FFRDCs.  The  policy 
requires  agencies  to  rely  on  existing  Government  research 
activities  and  private  industry  when  practicable  to  satisfy 
special  research  or  development  requirements.  DoD  sponsors  a 
total  of  11  FFRDCs  costing  about  $1.5  billion  annually. 

Objectives.  The  objectives  of  the  audit  that  relate  to  the 
OFPP  requests  were  to  determine  whether  DoD  sponsors  were 
performing  adequate  reviews  of  the  need  for  their  FFRDCs,  to 
determine  the  adequacy  of  the  sole-source  renegotiation 
justifications  for  the  FFRDC  contracts,  and  to  evaluate 
applicable  internal  controls.  A  future  report  will  address  the 
remainder  of  the  requested  work  from  the  OFPP  and  the  House 
Committee  on  Appropriations  request. 

Audit  Results.  DoD  sponsors  did  not  perform  adequate  reviews  and 
did  not  docxament  the  special  technical  needs  and  mission 
requirements  for  the  10  FFRDCs.  As  a  result,  the  bases  for  the 
most  recent  sole— source  contracts  to  the  FFRDCs  were  not 
adequate.  In  addition,  the  Navy  was  using  field  analysts  from 
the  Center  for  Naval  Analyses  without  determining  whether  that 
was  the  most  efficient  and  effective  use  of  resources. 

Internal  controls.  DoD  sponsors  did  not  follow  established 
policies  and  procedures  for  performing  comprehensive  reviews  of 
FFRDCs  and  justifying  renewal  of  their  sole-source  contracts.  We 
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considez*  the  weakness  to  be  material.  See  Part  I  for  a  summary 
of  internal  controls  reviewed  and  Part  II  for  details  of  the 
weakness . 

Potential  Benefits  of  Audit.  Benefits  should  result  from  better 
reviews  of  the  need  for  FFRDCs.  However,  we  could  not  quantify 
those  monetary  benefits.  About  $6.2  million  of  costs  could  be 
eliminated  if  the  Navy  replaced  the  41  Center  for  Naval  Analyses 
field  analysts  with  in-house  personnel.  The  potential  benefits 
of  the  audit  are  described  in  Appendix  R. 

Summary  of  Recommendations.  We  recommended  that  DoD  perform  new 
comprehensive  reviews,  establish  procedures  and  criteria  for  the 
reviews,  curtail  renewal  of  contracts  pendinq  new  reviews,  and 
perform  cost  and  effectiveness  analyses  on  the  use  of  Center  for 
Naval  Analyses  field  analysts. 

Manaqement  Comments.  The  Deputy  Director,  Defense  Research  and 
Enqineerinq;  the  Deputy  Assistant  Secretary  of  Defense  (Strateqic 
and  Tactical  Command,  Control,  and  Communications);  the  Office  of 
the  Assistant  Secretary  of  Defense  (Production  and  Loqistics) 
(now  Office  of  the  Deputy  Under  Secretary  of  Defense  (Loqistics) ; 
the  Army;  the  Navy;  the  Air  Force;  and  the  Director,  Advanced 
Research  Projects  Aqency,  did  not  aqree  that  prior  comprehensive 
reviews  and  justifications  for  the  sole-source  FFRDC  contracts 
were  inadequate.  They  aqreed  that  a  need  existed  to  improve  the 
process  for  comprehensive  reviews  and  to  either  perform  or  plan 
new  reviews.  The  Navy  disaqreed  that  Center  for  Naval  Analyses 
field  analysts'  effort  could  be  converted  to  in-house  work.  The 
Director  of  Defense  Procurement  disaqreed  with  a  recommendation 
to  require  synopsizinq  FFRDC  requirements. 

A  discussion  of  the  responsiveness  of  manaqement  comments  on  the 
recommendations  is  in  Part  II  of  the  report.  A  discussion  of 
comments  on  the  findinq  and  evaluations  of  comprehensive  reviews 
is  in  Part  III  of  the  report.  The  complete  text  of  manaqement 
comments  is  in  Part  IV  of  the  report. 

Audit  Response.  Based  on  management  comments,  we  revised  four 
report  recommendations.  Further,  we  deleted  the  recommen¬ 
dation  on  synopsizinq  requirements  and  added  a  recommendation  on 
clarifying  the  role  of  the  Director ,  Defense  Research  and 
Engineering,  in  sponsor  comprehensive  reviews.  Comments  are 
requested  from  the  Director,  Defense  Research  and  Engineering, 
and  the  Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition)  by  January  4,  1994. 
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PART  I  -  INTRODUCTION 


Background 

Federally  funded  research  and  development  centers  (FFRDCs)  were 
first  established  during  World  War  II  for  the  expeditious 
accomplishment  of  some  research  and  development  objectives  that 
required  assembling  scientific  talents  with  supporting  technical 
skills  to  work  closely  under  single  management.  The  initial 
contracts  were  awarded  for  development  of  nuclear  energy 
(Manhattan  Project) ,  for  development  of  effective  proximity  fuses 
for  anti-aircraft  ammunition  (John  Hopkins  University  Applied 
Physics  Laboratory) ,  and  for  research  in  rockets  (Jet  Propulsion 
Laboratory  of  the  California  Institute  of  Technology) .  The 
contributions  of  university-associated  scientists  and  engineers 
to  military  technology  encouraged  postwar  interest  in  maintaining 
research  centers  managed  by  universities  and  business  firms  under 
contract  to  the  Government. 

Since  World  War  II,  DoD  has  sponsored  as  many  as  18  FFRDCs  and  as 
few  as  6  FFRDCs.  Currently,  DoD  sponsors  11  FFRDCs  costing  about 
$1.5  billion  annually.  FFRDCs  perform  three  categories  of  work: 
studies  and  analyses,  systems  engineering  and  integration,  and 
laboratory  research  and  development.  Appendix  A  lists  the 
10  FFRDCs  we  audited,  their  primary  DoD  sponsors,  and  their 
authorized  funding. 

All  of  the  DoD-sponsored  FFRDCs  share  the  following 
characteristics . 

o  Each  is  managed  by  a  private,  non-profit  organization 
under  a  long-term  contract,  which  is  generally  for  5  years  and 
which  is  awarded  and  reawarded  without  competitive  procurement 
procedures . 

o  Each  is  closely  identified  with  a  sponsor,  either  a 
Military  Department  or  the  Office  of  the  Secretary  of  Defense 
(OSD) . 

o  Each  has  contractual  responsibilities  for  furthering  the 
interests  of  its  sponsor  in  mission  or  program  terms. 

o  The  personnel  policies  of  each  are  not  controlled  by 
Federal  civil  service  regulations,  salaries,  and  procedures. 
Personnel  restraints  exist,  but  they  are  contractual. 

o  The  DoD  sponsor  has  determined  that  Government 
requirements  cannot  be  effectively  accomplished  by  conventional 
contractual  arrangements  or  by  Government  personnel. 


FFRDCs  are  granted  access,  beyond  that  which  is  coroinon  in  a 
normal  contractual  relationship,  to  Government  and  supplier  data, 
including  sensitive  and  proprietary  data,  and  to  DoD  and  Defense 
contractor  employees  and  facilities.  This  unique  relationship 
requires  FFRDCs  to  operate  in  the  public  interest  with 
objectivity  and  independence,  to  be  free  from  organizational 
conflicts  of  interest,  and  to  fully  disclose  their  affairs  to  the 
DoD  sponsor. 

Office  of  Federal  Procurement  Policy  (OFPP)  Letter  84-1, 
"Federally  Funded  Research  and  Development  Centers"  (Appendix  B) , 
as  implemented  by  Federal  Acquisition  Regulations  (FAR)  35.017, 
"Federally  Funded  Research  and  Development  Centers"  (Appendix  C) , 
provides  policy  guidance  for  the  establishment,  use,  periodic 
review,  and  termination  of  FFRDCs.  The  OFPP  policy  provides 
that: 

Agencies  will  rely,  to  the  extent  practicable,  on 
existing  in-house  and  contractor  sources  for 
satisfying  their  special  research  or  development  needs 
consistent  with  established  procedures  under  The 
Economy  Act  of  1932  (31  U.S.C.  1535),  other 
statutory  authority  or  procurement/assistance 
regulations.  A  thorough  assessment  of  existing 
alternative  sources  for  meeting  these  needs  is 
especially  important  prior  to  establishing  an  FFRDC. 

The  Competition  in  Contracting  Act  of  1984 ,  which  was  enacted  to 
enhance  competition  and  restrict  unnecessary  sole-source 
contracting,  requires  that  all  Federal  contracts  be  competed, 
with  certain  exceptions.  One  exception.  United  States  Code, 
title  10,  section  2304(c)(3),  authorizes  the  use  of  other  than 
full  and  open  competition  when  maintaining  an  essential  FFRDC 
research  capability  is  necessary.  This  exception  is  implemented 
by  FAR  6.302-3,  "Industrial  Mobilization;  or  Engineering, 
Developmental,  or  Research  Capability,"  and  must  be  supported  by 
written  justifications  prepared  in  accordance  with  FAR  6.303, 
"Justifications,"  and  FAR  6.304,  "Approval  of  the  Justification," 
and  by  comprehensive  reviews  performed  in  accordance  with 
FAR  35.017-4,  "Reviewing  FFRDCs." 

Objectives 

The  audit  was  initially  requested  by  the  Administrator,  OFPP 
(Appendix  D) .  The  objectives  of  the  audit  were  to: 

o  determine  whether  DoD  sponsors  were  performing  adequate 
reviews  of  the  need  for  their  FFRDCs, 

o  determine  the  adequacy  of  the  sole-source  renegotiation 
justifications  for  the  FFRDC  contracts. 
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and 


o  assess  the  nature  and  extent  of  the  use  of  DoD  FFRDCs, 


o  evaluate  applicable  internal  controls. 

Based  on  direction  by  the  House  Committee  on  Appropriations  in 
House  Report  No.  102-95,  "Department  of  Defense  Appropriations 
Bill,  1992,"  we  added  audit  objectives  to: 

o  determine  whether  FFRDCs  adhered  to  mission  statements 
and  sponsoring  agreements, 

o  determine  whether  criteria  used  to  develop  overhead  rates 
for  the  FFRDCs  were  in  accordance  with  Government  standards,  and 

o  determine  whether  any  violations  of  conflict  of  interest 
regulations  existed  either  within  the  FFRDC  operation  or 
structure  or  in  the  DoD  relationship  with  the  FFRDC. 

This  report  addresses  the  first  and  second  objectives  on  adequacy 
of  reviews  and  the  sole-source  justifications  and  the  internal 
controls  related  to  both.  The  other  objectives  will  be  addressed 
in  a  subsequent  report. 

Scope 

We  evaluated  the  reviews  and  sole-source  justifications  prepared 
by  the  DoD  sponsors  for  contract  actions  occurring  in  FYs  1988, 
1989,  1990,  and  1991;  sponsoring  agreements;  procurement  and 
contract  documents;  mission  statements;  and  funding  data  for 
10  FFRDCs  that  existed  more  than  5  years.  The  10  are 
Project  AIR  FORCE;  National  Defense  Research  Institute;  Arroyo 
Center;  Institute  for  Defense  Analyses;  Center  for  Naval 
Analyses;  Logistics  Management  Institute;  Aerospace  Corporation; 
MITRE  Corporation  Command,  Control,  Communications  and 
Intelligence  (C^I)  Division;  Lincoln  Laboratory;  and  Software 
Engineering  Institute.  One  FFRDC,  the  Institute  for  Advanced 
Technology,  is  separately  discussed  in  Inspector  General,  DoD, 
Report  No.  93-013,  "Quick -Reaction  Report  on  the  Audit  of  the 
Army  Contract  with  the  University  of  Texas  at  Austin  Institute 
for  Advanced  Technology,"  October  27,  1992. 

We  also  considered  the  results  of  prior  audits,  congressional 
studies  and  testimony,  and  a  Navy  Procurement  Management  Review. 
We  interviewed  program  officials  in  the  sponsoring  DoD 
organizations,  DoD  contracting  officers,  FFRDC  users,  and  FFRDC 
management  officials.  Due  to  security  considerations,  our 
evaluation  did  not  include  the  reviews  or  sole-source 
justifications  for  the  renewal  of  contracts  awarded  by  the 
National  Security  Agency  to  the  Institute  for  Defense  Analyses. 
This  exclusion  does  not  affect  the  results  of  our  audit. 
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This  program  results  audit  is  based  on  work  performed  from 
October  1991  to  February  1993.  The  audit  was  made  in  accordance 
with  auditing  standards  issued  by  the  Comptroller  General  of  the 
United  States,  as  implemented  by  the  Inspector  General,  DoD. 
Accordingly,  we  included  such  tests  of  Internal  controls  as  were 
considered  necessary.  We  did  not  rely  on  any^  computerized  data 
to  conduct  this  part  of  the  audit.  The  activities  visited  or 
contacted  during  the  audit  are  listed  in  Appendix  S. 

Internal  controls 

We  evaluated  compliance  by  OSD  and  the  Military  Departments  with 
the  criteria  in  OFPP  and  OSD  guidance.  We  used  the  guidance  in 
OFPP  Memorandum  89-21,  ’'Improving  Management  Controls  Over 
Government  Procurement,"  July  17,  1989,  and  Under  Secretary  of 
Defense  for  Acquisition  memorandum,  "Certification  of  Procurement 
Systems,"  September  15,  1989,  to  evaluate  the  award  of  sole- 
source  contracts  to  the  10  FFRDCs  that  existed  more  than  5  years. 

The  audit  identified  a  material  internal  control  weakness  as 
defined  by  Public  Law  97-255,  Office  of  Management  and  Budget 
Circular  A— 123,  and  DoD  Directive  5010.38.  Policies  and 
procedures  require  DoD  sponsors  to  comprehensively  review  and 
document  the  use  of  and  need  for  FFRDCs.  Preparation,  review, 
and  approval  policies  and  procedures  also  exist  for 
justifications  of  sole-source  contracts.  However,  internal 
administrative  controls  were  not  followed  to  verify  that  adequate 
comprehensive  reviews  of  FFRDCs  were  completed  before  the 
justifications  to  renew  sole-source  contracts  for  FFRDCs. 
Implementation  of  Recoitonendation  1.  to  perform  new  comprehensive 
reviews  should  correct  the  weakness.  We  could  not  determine  the 
monetary  benefits  to  be  realized  by  implementing  Recommendation 
1.  because  the  benefits  will  depend  on  future  actions.  A  copy  of 
the  report  will  be  provided  to  the  senior  internal  control 
officials  in  OSD,  the  Military  Departments,  and  the  Advanced 
Research  Projects  Agency. 

Prior  Audits  and  Other  Reviews 

During  the  last  5  years,  the  Inspector  General,  DoD;  the  Air 
Force  Audit  Agency;  the  Office  of  the  Assistant  Secretary  of  the 
Navy  (Research,  Development,  and  Acquisition) ;  the  General 
Accounting  Office;  the  Congressional  Research  Service,  Library  of 
Congress;  and  the  Senate  Subcommittee  on  Oversight  of  Government 
Management,  Committee  on  Governmental  Affairs,  issued  ten  reports 
addressing  requirements  and  cost-effectiveness  issues  involving 
the  DoD-sponsored  FFRDCs.  Appendix  E  summarizes  those  reports. 
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PART  II  -  PIKDING  2^  RECOMMENDATIONS 


BASIS  FOR  SOLE-SOPRCE  COMTRACT  RENEWALS 

The  DoD  sponsors  did  not  adequately  document  the  basis  for 
renewing  the  10  FFRDC  sole-source  contracts  we  reviewed.  The  DoD 
sponsor  comprehensive  reviews  did  not  meet  OFPP  Letter  84-1  and 
FAR  35.017-4  requirements,  and  the  sole-source  justifications  for 
renewal  of  the  FFRDC  contracts  contained  unproven  statements. 
Three  reviews  were  not  done,  six  reviews  were  done  after  approval 
of  the  sole-source  justifications  and  were  not  adequate,  and  one 
review  was  incomplete.  The  inadequate  reviews  and  sole-source 
justifications  occurred  because  senior  officials  in  the  OSD,  the 
Military  Departments,  and  the  Advanced  Research  Projects  Agency 
and  the  responsible  contracting  officers  did  not  challenge  the 
timeliness,  completeness,  and  accuracy  of  the  comprehensive 
reviews  and  sole-source  justifications.  Thus,  the  reasons  used 
to  justify  the  sole-source  contracts  for  the  10  FFRDCs  were  not 
adequately  supported  by  documentation.  The  DoD  FFRDC  Management 
Plan,  which  is  being  implemented  and  is  an  improvement,  does  not 
go  far  enough  to  correct  the  review  and  justification 
documentation  problems.  Also,  the  Navy  can  reduce  costs  by 
$6.2  million  over  2  years  by  replacing  Center  for  Naval  Analyses 
field  analysts  with  Navy  employees. 

DISCUSSION  OF  DETAILS 


Ba  Clear  ound 

OFPP  Letter  84-1  contains  policy  guidance  on  FFRDCs  that  is 
implemented  in  DoD  by  the  FAR.  FAR  35.017-4  requires  the  DoD 
sponsor  to  conduct  a  comprehensive  review  of  the  use  of  and  need 
for  an  FFRDC  before  renewing  a  contract  or  agreement  for  the 
FFRDC.  The  head  of  the  sponsoring  organization  should  decide  to 
renew  or  terminate  the  FFRDC  arrangement  based  on  the  results  of 
the  comprehensive  review.  The  comprehensive  review  includes: 

o  an  examination  of  the  sponsor's  special  technical  needs 
and  mission  requirements  performed  by  the  FFRDC  to  determine 
whether,  and  at  what  level,  they  continue  to  exist; 

o  consideration  of  alternative  contractor  or  Government 
sources  to  meet  the  sponsor's  needs; 

o  an  assessment  of  the  FFRDC' s  efficiency  and  effectiveness 
in  meeting  the  sponsor's  needs,  including  the  FFRDC 's  ability  to 
maintain  its  objectivity,  independence,  quieJe-response 
capability,  currency  in  its  field(s)  of  expertise,  and 
familiarity  with  the  needs  of  its  sponsor; 

o  an  assessment  of  the  adequacy  of  the  FFRDC  management  in 
ensuring  a  cost-effective  operation;  and 
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o  a  determination  that  the  criteria  for  establishing  the 
FFRDC  continue  to  be  satisfied  and  that  the  sponsoring  agreement 
complies  with  FAR  35.017-1,  "Sponsoring  Agreements." 

FAR  35.017-4  requires  the  DoD  sponsor  to  coordinate  the 
FFRDC  review  with  other  users,  and  to  afford  the  other  users  an 
opportunity  to  assume  sponsorship  if  the  current  sponsorship  is 
no  longer  appropriate.  The  comprehensive  review  must  be  fonnally 
documented  since  the  review  documentation  provides  the  basis  for 
renewing  or  terminating  a  sponsor's  contractual  relationship  with 
the  FFRDC. 

Sufficiency  of  Reviews  and  Justifications 

The  reviews  and  justifications  for  the  10  FFRDCs  did  not 
adequately  document  the  5  areas  required  by  the  FAR  and  OFPP 
Letter  and  did  not  adequately  justify  renewal  of  the  sole-source 
contracts.  The  reviews  and  justifications  did  not  sufficiently 
document  the  sponsors'  mission  requirements  to  be  performed  by 
the  FFRDCs,  the  alternatives  considered  to  meet  sponsor 
requirements,  the  efficiency  and  effectiveness  of  the  FFRDCs  in 
meeting  sponsor  needs,  the  management  controls  established  by  the 
FFRDCs  to  ensure  cost-effective  operations,  and  the  determination 
that  criteria  for  establishing  the  FFRDCs  were  still  valid  and 
that  adequate  sponsoring  agreements  existed. 

A  comprehensive  review  document  addressing  the  OFPP  and  FAR 
guidance  on  the  continued  need  for  the  FFRDC  was  not  prepared  by 
the  DoD  sponsor  for  three  FFRDCs  and  was  prepared  after  the 
sole-source  justification  for  six  FFRDCs.  Of  the  six, 
two  comprehensive  reviews  were  not  approved  until  after  the 
contracts  were  awarded.  The  DoD  sponsor  prepared  the  review  for 
the  tenth  FFRDC  before  the  sole-source  justification  but  the 
documentation  did  not  adequately  address  the  OFPP  and  FAR 
criteria.  Appendix  F  summarizes  the  comprehensive  review  and 
sole-source  justification  approval  dates,  the  contract  award 
dates,  and  the  final  approval  level  for  the  awards  to  the 
10  FFRDCs. 

The  following  paragraphs  discuss  our  evaluation  of  the  reviews 
and  justifications.  Appendix  G  summarizes  the  deficiencies 
associated  with  the  reviews  and  justifications  for  each  of  the 
FFRDCs.  Appendixes  H  through  O  provide  details  on  our  assessment 
of  the  reviews  and  justifications  prepared  for  each  of  the 
10  DoD  FFRDCs. 

Sponsor  Mission  Requirements  for  FFRDCs.  The  comprehensive 
review  documents  did  not  detail  the  future  requirements  for  any 
of  the  10  FFRDCs  in  terms  of  specifically  who  needed  what 
services,  where,  when,  and  how.  The  reviews  did  not  detail  or 
assess  the  types  of  requirements,  such  as  long-term  program- 
oriented  research  versus  short-term  project/task  oriented  work. 
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or  the  extent  of  such  work.  They  did  not  identify  the  special 
skills  or  capabilities  needed  to  meet  the  requirements  or  the 
interrelationship  of  the  requirements  to  one  another .  ^  The 
requirements  were  stated  in  general  terms,  such  as  "a  continuing 
effort  for  studies  exists.”  Program  officials  who  performed  the 
reviews  found  that  users  were  uncertain  of  what  studies^  or 
support  would  be  needed  from  the  FFRDCs.  The  program  officials 
accepted  the  general  responses  rather  than  following  up  and 
insisting  that  users  identify  more  specific  requirements.  The 
comprehensive  reviews'  failure  to  detail  the  requirements  would 
hinder  subsequent  efforts  by  contracting  officials  to  identify 
and  consider  alternative  sources  capable  of  meeting  the 
requirements,  in  whole  or  part,  being  provided  by  the  FFRDCs. 
The  sole-source  justifications  described  in  general  terms  the 
services  to  be  procured  such  as  broad  categories  of  support 
services  or  studies  that  would  address  general  issues  or  areas. 

Alternative  contractor  and  Government  sources. ^  Statements 
in  the  review  documents  and  sole-source  justifications  that  no 
other  sources  existed  were  not  adequately  documented.  Formal 
market  surveys  were  not  performed  for  any  of  the  10  FFRDCs  to 
determine  whether  private-sector  firms,  in-house  Government 
personnel,  or  other  FFRDCs  could  perform  portions  of  the  work 
even  though  users  stated  that  alternatives  to  the  FFRDCs  did 
exist . 


GQiniiieT»ce  Business  Daily  synopses.  The  proposed  sole- 
source  contract  awards  for  4  of  the  10  FFRDCs  were  synopsized  in 
the  Commerce  Business  Daily.  The  Defense  Supply  Service- 
Washington  (DSS-W)  placed  a  notice  in  the  Commerce  Business  Daily 
August  17,  1988,  for  sole-source  contract  award  to  the  Institute 
for  Defense  Analyses;  June  29,  1989,  for  the  National  Defense 
Research  Institute;  August  8,  1989,  for  the  Logistics  Management 
Institute;  and  July  6,  1990,  for  the  Arroyo  Center. 

DSS-W  published  the  synopses  to  provide  public  notice  of  the 
pending  contract  awards.  The  synopses  were  neither  intended  nor 
worded  to  solicit  competition  on  the  procurements.  However,  in 
response  to  publicizing  the  contract  award  to  the  Institute  for 
Defense  Analyses,  nine  commercial  organizations  expressed  written 
interest  in  competing  for  the  contract  requirements.  On 
September  9,  1988,  and  October  6,  1988,  the  DSS-W  contracting 
officer  sent  letters  to  eight  of  the  nine  organizations 
reaffirming  DSS-W' s  intent  to  award  a  sole-source  contract  to  the 
Institute  for  Defense  Analyses.  The  DSS-W  letters  requested  the 
organizations  to  submit  a  "capability  statement"  setting  forth 
information  that  the  organizations  believed  would  qualify  their 
organization  as  an  FFRDC.  On  September  22,  1988,  the  DSS-W 
contracting  officer  sent  a  letter  to  the  ninth  organization, 
which  submitted  a  capability  statement  with  its  inquiry,  stating 
that  employing  other  sources  to  meet  the  requirements  would  not 
be  cost  effective.  There  was  no  evidence  that  sponsoring  program 
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officials  or  the  contracting  officer  ever  evaluated  the 
organization's  capability.  None  of  the  organizations  pursued  the 
matter  further . 

FAR  5.205(b),  "Special  Situations,”  provides  that  advance  notices 
in  the  Commerce  Business  Daily  and  Federal  Register  are  only 
required  before  establishing  an  FFRDC  or  before  changing  the 
FFRDC  basic  purpose  or  mission,  and  that  a  notice  is  not  required 
where  a  procurement  action  is  required  by  law.  Also, 

FAR  5.202 (a) (10) ,  "Exceptions,”  states  that  a  contracting  officer 
need  not  submit  a  notice  of  proposed  contract  action  to  the 
Commgrce  Business  Daily  when  the  contract  action  is  made  under 
FAR  6.302-3,  and  advance  notice  is  not  appropriate  or  reasonable. 
Contracting  officers  cited  FAR  6.302-3  in  the  sole-source 
justifications  as  the  authority  for  awarding  sole-source 
contracts  to  FFRDCs. 

Regarding  future  acquisitions,  FAR  6.303,  Justifications," 
paragraph  6. 303-2 (a) (11) ,  "Content,"  states  that  sole-source 
justifications  are  required  to  .include,  "A  statement  of  the 
actions,  if  any,  the  agency  may  take  to  remove  or  overcome  the 
barriers  to  competition  before  any  subsequent  acquisition  for  the 
supplies  or  services  required."  In  addressing  this  requirement, 
contracting  officers  again  cited  FAR  6.302-3  in  7  of  the  10  sole- 
source  justifications  as  authority  for  not  considering 
alternatives  that  would  mitigate  the  need  for  future  sole-source 
reliance  on  FFRDCs.  The  remaining  three  justifications  stated 
that  alternatives  would  be  considered  during  the  next 
comprehensive  review. 

Congressional  hearings.  According  to  testimony  on 
systems  development  and  management  within  the  Federal  Government 
given  during  hearings  before  the  House  Committee  on  Government 
Operations  in  June  through  August  1962,  FFRDCs  were  originally 
established  because  DoD  could  not  attract  and  retain  scientists 
and  engineers.  At  that  time.  Government  pay  was  not  comparable 
with  the  private  sector.  The  President's  Scientific  Adviser 
stated  during  these  hearings  that: 

...it  is  very  important  that  we  make  a  major  effort,  a 
conscious  effort,  to  increase  our  own  in-house 
capability  to  do  these  things.... if  we  can  do  that, 
then  we  can  slowly  get  away  from  the  necessity  for 
these  ad  hoc  mechanisms. ..  .if  you  examine  the  temper 
of  those  earlier  times,  particularly  during  the  early 
days  of  the  ballistic  missile  effort,  there  was  a  very 
real  crisis ....  Suddenly  there  was  the  danger  that 
another  nation  would  have  them  and  we  would 
not.... Most  governmental  research  and  development 
problems  are  not  of  that  nature. 

None  of  the  reviews  or  sole-source  justifications  for  the 
10  FFRDCs  cited  low  Federal  salaries  for  scientific  and  technical 
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positions  as  a  reason  why  the  Federal  Government  could  not 
attract  the  scientific  and  technical  personnel  to  perform 
required  tasks  in-house  today. 

Independence.  Independence  and  objectivity  were  cited 
as  qeneral  reasons  why  private— sector  firms  could  not  perform  the 
tasks  of  the  FFRDCs.  However,  the  reviews  and  justifications  did 
not  address  why  other  nonprofit  organizations,  including  some 
former  DoD  FFRDCs,  would  lack  independence  and  objectivity. 

Assessment  of  efficiency  and  effectiveness^  DoD  sponsors 
determined  FFRDC  efficiency  and  effectiveness  based  on 
questionnaires  completed  by  users,  either  for  individual  projects 
or  for  all  projects  worked  during  a  particular  period.  Overall, 
the  users  rated  the  10  FFRDCs  as  responsive  to  user  needs.  The 
users  noted  problems  with  financial  reporting  and  completion 
delays  on  some  taskings,  but,  overall,  users  were  satisfied  with 
the  FFRDC  services. 

We  reviewed  the  appropriateness  of  the  DoD  sponsors'  methods  for 
evaluating  FFRDC  efficiency  and  effectiveness  in  meeting  user 
needs.  We  also  reviewed  the  reliability  of  the  data  reported  in 
user  assessments  and  summarized  in  the  comprehensive  reviews  by 
the  primary  DoD  sponsors.  The  user  assessments  of  the  FFRDCs 
included  ratings  that  were  generally  adjective  descriptions 
rather  than  quantitative  ratings  based  on  objective  criteria. 
The  assessment  forms  requested  check -mark  answers,  narrative 
comments,  or  both,  but  contained  little,  if  any,  supporting 
details.  Critical  remarks  made  by  users  for  6  of  10  FFRDCs  were 
ignored  or  distorted  in  the  summary  comments  prepared  by  the 
primary  DoD  sponsors  and  were  not  included  in  the  overall 
comprehensive  reviews.  We  concluded  that  user  responses  and 
primary  DoD  sponsor  assessments  of  the  FFRDCs'  efficiency  and 
effectiveness  in  the  comprehensive  reviews  were  often  personal 
opinion.  Procedures  were  needed  for  the  sponsors  of  FFRDCs  to 
document  measurement  of  the  efficiency  and  effectiveness^  of 
FFRDCs  in  a  consistent  or  a  standardized  manner.  In  addition, 
the  procedures  should  cover  followup  on  the  comments  critical  of 
FFRDC  work. 

Assessment  of  FFRDC  management  controls.  Program  officials 
who  prepared  the  summary  comprehensive  review  documents  did  not 
adequately  document  their  assessment  of  management  controls  for 
any  of  the  DoD  FFRDCs.  The  review  documents  generally  stated 
that  DoD  administrative  contracting  officers,  the  Defense 
Contract  Audit  Agency,  and  DoD  policy  and  advisory  groups  that 
provided  oversight  to  each  FFRDC  verify  that  FFRDC  operations 
were  cost-effective.  None  of  the  reviews  documented  the  internal 
steps  FFRDCs  took  to  verify  that  accurate  accounting  records  were 
maintained,  that  sound  procurement  practices  were  followed,  that 
overhead  costs  were  reasonable,  or  that  management  fees  were 
appropriate  and  necessary  to  the  FFRDC. 
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Complianea  with  PgRDG  criteria.  The  review  documents  stated 
that  criteria  for  establishing  the  FFRDCs  were  satisfied  because 
sponsor  mission  reguirements  continued  and  because  senior  DoD 
officials  agreed  that  contracts  should  be  renewed.  The  review 
documents  did  not  specifically  address  the  sponsoring  agreements. 

Establishment  criteria.  As  required  by  FAR  35.017-2, 
"Establishing  or  Oianging  an  FFRDC,"  none  of  the  review  documents 
for  the  10  FFRDCs  adequately  established  that: 

o  the  mission  of  the  FFRDC  was  stated  clearly 
enough  to  differentiate  between  work  that  should  be  performed  by 
the  FFRDC  and  work  that  should  be  performed  by  non-FFRDCs, 

o  conventional  contracting  methods  and  existing 
in-house  facilities  could  not  effectively  satisfy  the  sponsor's 
special  research  and  development  requirements,  and 

o  controls  were  in  place  to  ensure  the  cost  of 
FFRDC  services  were  reasonable. 

Sponsoring  agreements.  The  comprehensive  reviews  for 
the  10  FFRDCs  did  not  specifically  address  whether  a  sponsoring 
agreement  was  maintained  in  accordance  with  FAR  35.017-1. 
However,  we  determined  that  the  FAR  35.017-1  requirements  for  the 
10  FFRDCs  were  included  in  the  sponsoring  agreements,  the 
contracts,  or  both.  FAR  35.017-1  states  that  a  sponsoring 
agreement  or  other  legal  instrument  must  include  a  mission 
statement,  provisions  for  the  orderly  termination  of  the  FFRDC, 
provisions  for  the  identification  and  use  of  retained  earnings,  a 
prohibition  against  the  FFRDC  competing  with  non-FFRDCs,  and  a 
statement  on  whether  the  FFRDC  may  accept  work  from  non-sponsors. 

Responsibility  for  Complete  and  Accurate  Justifications 

FAR  6.303  states  that  technical  and  requirements  personnel  are 
responsible  for  providing  and  certifying  as  accurate  and  complete 
the  necessary  data  to  support  their  recommendation  for  other  than 
full  and  open  competition.  FAR  6.303  further  states  that 
contracting  officers  shall  not  commence  negotiations  for  sole- 
source  contracts  until  the  contracting  officers  justify  the  use 
of  a  sole-source  contract  in  writing  and  certify  to  the  accuracy 
and  completeness  of  the  justification.  FAR  1.602,  "Contracting 
Officers,”  paragraph  1. 602-1 (b),  "Authority,"  states  that 
contracting  officers  shall  enter  into  no  contracts  unless  all 
requirements  of  law,  executive  orders,  regulations,  and  other 
applicable  procedures,  including  clearances  and  approvals,  have 
been  met.  FAR  6.304  requires  the  head  of  the  procuring  activity 
to  approve  the  justification  if  the  proposed  contract  is  between 
$1  million  and  $10  million.  The  senior  procurement  executive  of 
the  procuring  activity  must  approve  a  proposed  contract  of  more 
than  $10  million. 
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Support  for  justifications.  The  written  reviews  did  not 
support  the  10  sole-source  justifications.  Technical  personnel, 
contracting  officers,  and  senior  acquisition  officials  certified 
to  the  accuracy  and  completeness  of  each  sole-source 
justification  although  comprehensive  reviews  for  9  of  the 
10  FFRDCs  were  not  completed  at  the  time  the  sole-source 
justifications  were  prepared.  The  tenth  review  was  complete  but 
was  inadequate.  None  of  the  seven  reviews  that  were  done 
addressed  all  five  criteria  from  the  OFPP  and  FAR  guidance  for 
retaining  an  FFRDC.  The  Under  Secretary  of  Defense  for 
Acquisition,  the  Service  Acquisition  Executives,  the  Director, 
Defense  Research  and  Engineering  (DDR&E) ,  and  the  contracting 
officers  should  have  challenged  the  accuracy  and  completeness  of 
the  determinations  in  the  sole-source  justifications^  and  should 
have  determined  whether  the  assertions  in  each  justification  were 
supported  by  a  comprehensive  review  document.  The  justifications 
did  not  contain  sufficient  facts  and  rationale  to  justify  the  use 
of  FAR  6.3  02-3  for  sole-source  awards  to  the  10  FFRDCs. 
FAR  6.302-3  should  be  cited  only  when  awarding  a  contract  to  a 
particular  source  is  necessary  to  establish  or  maintain  an 
essential  engineering,  research,  or  development  capability 
provided  by  an  educational  or  other  non-profit  institution  or  an 
FFRDC. 

There  was  no  evidence  that  the  contracting  officers  or  senior 
acquisition  officials  ever  requested  additional  documentation  or 
rationale  for  the  sole-source  contracts  before  approving  the 
sole-source  justifications.  Statutory  and  regulatory  provisions 
require  adequate  assertions  to  justify  noncompetitive 
procurements.  Comprehensive  reviews  that  were  not  completed  or 
were  not  adequately  documented  would  not  be  a  basis  the 
required  assertions.  Contracting  officers  and  senior  acquisition 
officials  are  responsible  for  maximizing  competition  and  should 
not  decide  that  the  sole-source  organization  was  the  only  one 
available  without  complete  and  adequately  docvimented 
comprehensive  reviews.  Appendix  P  identifies  the  officials  and 
offices  responsible  for  preparing  and  approving  the  sole-source 
justifications. 

Future  awards.  The  Military  Departments  or  DSS-W  should  not 
award  a  new  5-year  sole-source  contract  to  any  FFRDC  until  an 
adequate  comprehensive  review  and  a  sole-source  justification  are 
completed.  FFRDC  services  that  can  not  be  properly  justified  in 
whole  or  in  part  should  be  discontinued  consistent  with  an 
orderly  transitioning  from  reliance  on  the  FFRDC.  DoD  sponsors 
should  compete,  or  perform  in-house,  any  future  mission 
requirements  or  technical  needs  that  can  be  more  effectively 
accomplished  by  not  using  the  FFRDC. 

Effect  of  Sole-Source  Contracts 

The  awarding  of  sole-source  contracts  to  operate  FFRDCs 
eliminated  competition  from  other  organizations  to  operate  the 
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FFRDCs,  and  permits  sponsors  to  exempt  subsequent  research  and 
development  tasks  to  the  FFRDCs  from  competition.  As  a  general 
rule,  requirements  should  be  competed  unless  the  requiring 
Government  activity  can  demonstrate  that  the  Government  would  be 
injured  by  a  critical  delay  or  would  incur  substantial  additional 
costs  or  that  competition  is  not  feasible.  Some  users  of  the 
FFRDCs,  in  providing  input  to  the  comprehensive  reviews,  stated 
that  alternative  contract  sources  could  have  performed  tasks 
assigned  to  the  FFRDCs.  Recent  Congressional  testimony  related 
to  DoD  authorizations  and  appropriations  for  FYs  1991,  1992, 
1993,  and  1994  noted  the  existence  of  capable  private-sector 
firms  that  are  interested  in  competing  for  portions  of  the  work 
tasked  to  FFRDCs.  The  Senate  Subcommittee  on  Oversight  of 
Government  Management  Report,  "Inadequate  Federal  Oversight  of 
Federally  Funded  Research  and  Development  Centers,"  July  8,  1992, 
concluded  that  FFRDC  operating  contracts  could  and  should  be 
competed  more  frequently  than  they  are. 

Evidence  also  exists  that  awarding  sole-source  contracts  resulted 
in  additional  costs  to  perform  the  work.  As  noted  in  Inspector 
General,  DoD,  Report  No.  91-115,  "Consulting  Services  Contracts 
for  Operational  Test  and  Evaluation,"  August  22,  1991,  auditors 
estimated  the  cost  for  Institute  for  Defense  Analyses  operational 
test  and  evaluation  support  was  about  31  percent  greater  than  if 
Government  employees  were  used.  Air  Force  Audit  Agency  Report 
No.  0056410,  "Review  of  Air  Force  Managed  Federally  Funded 
Research  and  Development  Centers,"  August  6,  1991,  stated  that 
the  average  cost  of  a  member  of  the  technical  staff  on  the 
FY  1990  Aerospace  Corporation  contract  was  about  $184,000, 
compared  to  about  $114,000  annually  for  a  non-FFRDC  technical 
support  contractor  employee  in  the  Los  Angeles  area.  The  average 
FY  1990  cost  for  a  member  of  the  technical  staff  at  MITRE  was 
about  $151,000,  whereas  the  comparative  cost  for  a  non-FFRDC 
contractor  in  the  Boston  area  was  about  $135,000.  In  collecting 
information  on  Center  for  Naval  Analyses  field  analysts,  we 
estimated  that  during  FYs  1994  and  1995,  the  Navy  could  eliminate 
about  $6.2  million  of  costs  by  converting  the  41  Center  for  Naval 
Analyses  field  analysts  to  Government  GM-15  analysts  (Appendix 
J). 

DoD  Directive  4205.2,  "Contracted  Advisory  and  Assistance 
Services,"  states  that,  when  an  in-house  capability  is  not 
available,  the  organization  should  establish  the  needed 
capability  as  soon  as  practicable  to  avoid  a  long-term  dependency 
on  contracting  support,  unless  establishing  this  capability  would 
not  be  cost  effective.  The  Air  Force  recently  initiated  a  test 
program,  CORAL  CONVERT,  to  reduce  reliance  on  FFRDC  services. 
The  program  calls  for  conversion  of  100  engineering  positions  at 
the  Aerospace  Corporation  and  50  engineering  positions  at  the 
MITRE  Corporation  C^I  Division  to  in-house  positions  at  Air  Force 
Materiel  Command's  Space  and  Missile  Systems  Center  and 
Electronic  Systems  Center,  respectively.  Currently,  the  Air  Force 
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estiiaates  savings  of  about  $31,000  for  each  converted  Aerospace 
Corporation  employee  and  about  $25,700  for  each  converted 
MITRE  Corporation  c^l  Division  employee.  The  test  program  was 
started  during  FY  1993  after  the  Air  Force  Materiel  Command 
obtained  waivers  from  civilian  hiring  freezes  and  authority  to 
reimburse  civilian  pay  with  research,  development,  test,  and 
evaluation  funds.  Initial  test  results  at  the  Electronic  Systems 
Center  indicate  that  capable  engineers  were  being  hired  in-house 
in  place  of  MITRE  Corporation  C^I  Division  engineers.  In  May 
1993,  the  Air  Force  Materiel  Command  extended  the  test  program 
through  December  1994  due  to  delays  in  implementing  the  test 
program  at  the  Space  and  Missile  Systems  Center.  On  September 
10,  1993,  the  Air  Force  Deputy  Assistant  Secretary  (Acquisition) 
suspended  the  test  program  pending  an  evaluation  of  the  impact  of 
the  Vice  President's  Report  of  the  National  Performance  Review  on 
staffing  levels  within  the  acquisition  community. 

DoD  FFRDC  Management  Plan 

On  August  14,  1992,  the  DDR&E  issued  a  memorandum, 
"Implementation  of  the  Federally  Funded  Research  and  Development 
Center  Management  Plan"  (the  Management  Plan) .  The  memorandum 
stated  that  the  primary  sponsor  should  develop  a  program  for  the 
work  of  the  FFRDC  during  the  next  fiscal  year  within  the 
established  funding  ceiling;  should  recommend  changes  to 
established  funding  ceilings;  should  review  the  cost  and  value  of 
goods  and  services  provided  by  the  FFRDC;  and,  before  renewal ^of 
the  sponsorship  agreement,  should  conduct  a  comprehensive  review 
of  the  use  of  and  need  for  the  FFRDC. 

DDR&E  requested  each  DoD  Component  sponsoring  an  FFRDC  to  provide 
by  October  15,  1992,  a  copy  of  the  DoD  Component's  directives 
issued  to  ensure  compliance  with  the  management  plan,  as  well  as 
a  copy  of  any  comprehensive  review  of  the  FFRDC  performed  since 
January  1,  1990.  He  also  requested  DoD  sponsors  to  complete  new 
comprehensive  reviews  by  October  15 ,  1992 ,  if  the  last 
comprehensive  review  was  performed  before  January  Ij  1990.  The 
offices  of  the  Assistant  Secretary  of  Defense  (C^I) ,  and  the 
Deputy  Chief  of  Naval  Operations  (Navy  Program  Planning) 
conducted  comprehensive  reviews  after  January  1,  1990,  for  the 
MITRE  Corporation  C^I  Division  and  the  Center  for  Naval  Analyses, 
respectively.  New  comprehensive  reviews  were  required  for 
Project  AIR  FORCE,  National  Defense  Research  Institute,  Arroyo 
Center,  Institute  for  Defense  Analyses,  Logistics  Management 
Institute,  Aerospace  Corporation,  Lincoln  Laboratory,  and  the 
Software  Engineering  Institute.  As  of  September  10,  1993,  the 
Deputy  DDR&E  approved  new  comprehensive  reviews  only  for  the 
MITRE  Corporation  c^l  Division  and  the  Institute  for  Defense 
Analyses .  We  did  not  review  the  adequacy  of  the  new 
comprehensive  reviews  for  the  MITRE  Corporation  C"*!  Division  or 
the  Institute  for  Defense  Analyses. 
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Manaaamen'b  Plan  guidance.  The  ^  DDR&E  developed  the 
Management  Plan  in  response  to  Congressional  concern  that  OSD 
should  exercise  more  control  over  the  growth  and  use  of  FFRDCs. 
Congress  noted  that  sponsors  tasked  FFRDCs  to  perform  work  that 
had  no  impact  on  national  security  or  on  the  ability  of  a 
particular  Service  to  accomplish  its  mission.  A  ^  principal 
control  mechanism  for  verifying  that  an  FFRDC  relationship  is 
properly  continued  and  that  an  FFRDC  is  only  ^  used  when 
appropriate  is  a  formally  documented  comprehensive  review 
prepared  under  the  guidelines  of  OFPP  Letter  84-1  and 
FAR  35.017-4. 

The  Management  Plan  re-emphasized  that  DoD  sponsors  should 
perform  comprehensive  reviews  in  accordance  with  OFPP 
Letter  84-1.  The  Management  Plan  stated  that  primary  sponsors 
will  conduct  the  comprehensive  reviews  with  DDR&E  participation 
and  that  sponsoring  agencies  should  not  decide  to  continue  or 
terminate  an  FFRDC  relationship  without  DDR&E  concurrence.  The 
Management  Plan  stated  no  additional  guidance  on  how  DoD  sponsors 
should  perform  the  comprehensive  reviews.  The  plan  also  did  not 
detail  the  purpose  or  extent  of  DDR&E  participation  in  the 
conduct  of  comprehensive  reviews  or  provide  criteria  for  DDR&E 
concurrence  with  sponsor  determinations  on  the  continued  need  for 
FFRDC  relationships.  The  Management  Plan  stated  that  DoD  would 
manage  FFRDCs  through  ceilings  on  the  funds  available  to  pay 
FFRDC  costs  and  the  work  years  for  FFRDC  technical  staff .  In 
recent  years,  the  funding  ceilings  were  established  by  the  annual 
Defense  Authorization  and  Appropriation  Acts.  The  Management 
Plan  also  required  the  sponsors  of  the  FFRDCs  to  report  annually 
to  the  DDR&E  on  the  costs  and  value  to  DoD  of  goods  and  services 
provided  by  each  FFRDC  and  the  reasonableness  of  the  costs.  We 
concluded  that  the  Management  Plan  improved  oversight  of  the 
FFRDCs,  but  provided  little  assurance  that  the  quality  of  the 
comprehensive  reviews  of  FFRDCs  would  be  improved.  Also,  the 
Management  Plan  will  not  ensure  that  FFRDC  relationships  will 
only  be  continued  for  special  technical  needs  that  can  not  be  met 
elsewhere. 

RECOMMENDATIONS.  MANAGEMENT  COMMENTS.  AND  AUDIT  RESPONSE 

gmnmarles  of  management  eommenta.  Management  comments  on 
the  recommendations  are  summarized  below.  Appendix  Q  summarizes 
management  comments  on  the  finding.  Appendixes  through  0 
summarize  management  comments  on  the  audit  evaluation  of  the 
comprehensive  reviews  and  FFRDC  contracts.  The  Assistant 
Secretary  of  Defense  (Production  and  Logistics)  comments  stated 
that  he  agreed  with  the  DDR&E  comments  in  total. 

Changes  to  raeommendations .  We  revised  four 
recommendations  based  on  management  comments  and  discussions  with 
DDR&E  officials.  Recommendation  l.b.  was  revised  to  clarify  our 
intent  to  improve  the  documentation  of  and  provide  consistency  to 
the  comprehensive  reviews  conducted  by  the  FFRDC  sponsors. 
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Recommendation  l.c.  was  revised  to  clarify  the  options  available 
to  FFRDC  sponsors  when  FFRDC  relationships  are  no  longer 
justified.  Recommendation  2.  was  revised  to  ensure  that  ongoing 
work  continues  while  comprehensive  reviews  are  conducted. 
Recommendation  3.  was  revised  to  recommend  that  the  Navy  consider 
all  factors,  including  costs,  when  analyzing  use  of  field 
analysts. 

Based  on  management  comments,  we  deleted  draft  Recommendations 
l.c.  and  4.  to  publish  advance  notices  in  the  Commerga  Pusiness 
Daily  before  awarding  renewal  contracts  to  FFRDCs-.  We  renumbered 
draft  Recommendation  l.d.  to  l.c.,  and  we  added  a  new 
Recommendation  l.d.  to  clarify  the  DDR&E  role  in  FFRDC  sponsor 
comprehensive  reviews. 

1,  jub  recommend  that  the  Director#  Defense  Research  and 
Engineering: 

a.  Direct  DoD  sponsors  for  all  PPRDCs  to  prepare  new 
comprehensive  reviews. 

Management  The  Deputy  DDR&E  partially  concurred 
with  the  recommendation  and  stated  that  new  comprehensive  reviews 
would  be  conducted  in  accordance  with  the  Management  Plan.  New 
comprehensive  reviews  would  not  be  reguested  for  the  MITRE 
Corporation  C^I  Division  and  Center  for  Naval  Analyses  because 
the  FFRDC  sponsors  had  completed  reviews  since  January  1,  1990. 

The  Deputy  Assistant  Secretary  of  Defense  (Strategic  and 
Tactical  C^)  concurred  and  stated  a  new  comprehensive  review  for 
MITRE  Corporation  C^I  Division  was  to  be  completed  by  June  15, 
1993. 

The  Army  nonconcurred  and  stated  that  the  comprehensive^  review 
for  the  Arroyo  Center  was  performed  in  accordance  with  the 
guidance  existing  before  contract  award  and  that  the  F^ 
comprehensive  review  requirsments  were  not  effective  until 
March  1990.  The  Army  also  stated  that  performance  of  a  new 
review  at  this  time  would  delay  future  work,  would  be  unfair  to 
FFRDC  planning,  and  could  adversely  affect  efforts  to  deal  with 
significant  defense  issues  in  a  timely  manner .  The  Army  added 
that  no  legitimate  legal  basis  for  conducting  comprehensive 
reviews  exists  at  this  time. 

The  Navy  nonconcurred  and  stated  the  comprehensive  review 
supporting  the  current  Center  for  Naval  Analyses  contract  was  in 
accordance  with  existing  guidance.  The  Navy  further  stated  that 
the  next  comprehensive  review  for  the  Center  for  Naval  Analyses 
was  scheduled  to  start  in  the  last  quarter  of  FY  1994,  and  that 
conducting  a  review  sooner  would  be  an  imprudent  use  of  limited 
resources.  The  Navy  estimated  that  about  1  year  would  be  needed 
to  perform  a  new  comprehensive  review. 


15 


The  Air  Force  concurred  and  stated  that  new  comprehensive  reviews 
would  be  conducted  at  all  Air  Force-sponsored  FFRDCs. 

The  Director,  Advanced  Research  Projects  Agency,  nonconcurred  and 
stated  that  a  comprehensive  review  for  the  Software  Engineering 
Institute  would  be  conducted  during  the  summer  of  ^93  as  a  part 
of  the  normal  5-year  contract  review  process  and  is  expected  to 
be  completed  by  June  30,  1994. 

Audit  Response .  Although  the  Deputy  DDR&E  did  not  direct 
initiation  of  new  comprehensive  reviews  at  the  MITRE 
Corporation  C^I  Division  and  the  Center  for  Naval  Analyses, 
the  comments  from  the  sponsors  of  those  two  FFRDCs  stated  a 
comprehensive  review  was  either  already  started  or  was 
planned  to  start  in  FY  1994.  The  cumulative  management 
actions  satisfy  the  intent  of  the  recommendation.^  We  ask 
the  Deputy  DDR&E  to  provide  the  expected  completion  dates 
for  all  FFRDC  comprehensive  reviews  in  response  to  the  final 
report . 

We  disagree  with  Army  and  Navy  statements  that  existing 
comprehensive  reviews  met  the  requirements  of 
OFPP  Letter  84-1  and  FAR  35.017.  Although  FAR  35.017  was 
not  implemented  when  the  Army  comprehensive  review  of  the 
Arroyo  Center  occurred,  the  requirements  of  OFPP  Letter  84-1 
had  been  in  effect  since  1984.  The  OFPP  Letter  84-1 
requirements  for  a  comprehensive  review  are  similar  to  the 
requirements  cited  in  the  March  1990  FAR  requirements. 
Further,  Army  and  Navy  comments  did  not  cite  additional 
information  to  change  our  conclusion  that  the  comprehensive 
reviews  previously  performed  were  not  adequately  documented 
and  were  not  complete  for  the  five  areas  subject  to  review. 

b.  Establish  procedures  for  sponsors  to  adequately  docximent 
conduct  of  the  comprehensive  reviews.  If  needed,  a  working  group 
of  the  sponsors  of  FFRDCs  should  draft  the  procedures.  The 
procedures  should  include  criteria  for  measuring  whether  a 
comprehensive  review  meets  the  intent  of  the  Office  of  Federal 
Procurement  Policy  Letter  and  the  Federal  Acqpiisition  Regulation. 
The  criteria  should  include  evaluations  of: 

(1)  The  special  research  and  development  needs,  the 
special  skills  and/or  capabilities  involved  in  meeting  the  needs, 
and  the  estimated  extent  of  the  needs. 

(2)  The  alternatives  for  accomplishing  the 
requirements,  to  include  additional  in— house  staffing, 
conventional  contracts,  and  other  DoD  and  non— DoD  FFRDCs.  The 
review  should  compare  the  cost  to  obtain  all  or  portions  of  the 
support  from  the  FFRDC  versus  the  cost  to  perform  the  work  with 
additional  Government  personnel. 
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(3)  The  objective  assessment  of  FFRDC  efficiency  and 
effectiveness  in  meeting  DoD  sponsor  needs,  to  include  the  number 
and  dollar  value  of  projects  and  programs  assessed  and  follow-up 
and  resolution  by  higher-level  management  of  all  project  and 
progr2Uii  assessments  that  were  critical  of  the  FFRDC. 

(4)  The  controls  established  by  the  FFRDC  to  verify 
cost-effective  operation.  Reviews  should  address  the  maintenance 
of  accurate  accounting  records,  the  following  of  sound 
procurement  practices,  the  reasonableness  of  overhead  costs a 
summary  of  any  Defense  Contract  Audit  Agency  and  administrative 
contracting  officer  reviews  and  any  related  corrective  actions, 
and  the  need  for  management  fees. 

(5)  The  assurances  that  criteria  for  establishing  an 
FFRDC  continue  to  be  satisfied  and  that  an  adequate  sponsoring 
agreement  exists. 

Manacemftwt  cQinments.  The  Deputy  DDR&E,  the  Deputy  Assistant 
Secretary  of  Defense  (Strategic  and  Tactical  C^) ,  the  Navy,  the 
Air  Force,  and  the  Director,  Advanced  Research  Projects  Agency, 
partially  concurred.  They  all  stated  that  comprehensive  reviews 
should  follow  the  existing  guidelines  in  OFPP  Letter  84-1  and 
FAR  35.017-4.  They  disagreed  with  the  additional  review 
requirements  proposed  by  the  recommendation. 

The  Army  stated  that  considering  additional  in-house  staffing 
versus  contracting  with  an  FFRDC  or  other  business  concern  might 
not  be  viable  in  light  of  current  military  and  civilian 
drawdowns.  Further,  the  use  of  additional  in-house  staffing  to 
perform  the  kind  of  studies  that  FFRDCs  perform  appears  far  from 
promising. 

The  Navy  stated  that  the  primary  sponsor  and  not  the  DDR&E  should 
determine  the  methodology  and  documentation  of  the  review.  The 
Navy  agreed  to  consider  the  recommendations  during  the  next 
comprehensive  review  of  the  Center  for  Naval  Analyses. 

The  Air  Force  stated  that  the  reviews  must  be  conducted  with  the 
understanding  that  FFRDCs  are  a  unique  national  resource 
deliberately  protected  from  normal  competition  and  that  FFRDCs 
support  the  military  in  complex  problems  related  to  critical 
aspects  of  national  security.  Further,  the  guidelines  for  a 
review  in  the  recommendation  were  not  the  same  as  those  in  the 
FAR,  and  the  FAR  guidelines  represent  a  better  balance  of  the 
necessity  to  protect  the  Government's  interests  while  recognizing 
the  unique  nature  of  FFRDCs. 

The  Director,  Advanced  Research  Projects  Agency,  stated  his 
organization  would  work  with  the  DDR&E  to  establish  evaluation 
criteria  consistent  with  the  FAR  and  will  document  the 
comprehensive  review  according  to  that  criteria. 
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Audit  response.  Based  on  management  comments,  we  revised 
the  recommendation  to  more  clearly  show  our  intent  to 
improve  the  documentation  and  consistency  of  sponsor 
comprehensive  reviews.  The  requirements  of  OFPP  Letter  84-1 
and  FAR  35.017-4  provide  only  general  guidelines,  not 
definitive  criteria,  on  the  content  of  ^  a  comprehensive 
review.  We  believe  that  the  reasonable  implementation  of 
the  OFPP  and  FAR  guidelines  should  address  the  issues  listed 
in  the  recommendation.  The  quality  and  consistency  of 
documentation  in  the  comprehensive  reviews  the  FFRDCs 
varied  greatly.  Guidance  is  needed  to  improve  the 
thoroughness  and  consistency  of  reviews.  We  agree  that 
FFRDCs  are  unique  and  have  supported  the  military  well  in 
resolving  complex  problems.  However,  past  practices  and 
circumstances  are  not  necessarily  a  roadmap  for  the  future. 
The  Army  should  note  that  the  Air  Force  recently  gained 
additional  in-house  staffing  to  replace  FFRDC  resources. 
Replacing  FFRDC  resources  with  in-house  staffing  should 
occur  when  using  in-house  staff  is  a  more  efficient  and 
effective  way  to  provide  the  support  and  arbitrary 
limitations  on  in-house  staffing  should  be  rejected. 

We  ask  that  the  Deputy  DDR&E  consider  the  need  for  more 
thorough  and  consistent  documentation  by  the  different 
sponsors  performing  comprehensive  reviews,  and  provide 
comments  on  the  revised  recommendation. 

c.  Establish  procedures  for  sponsors  of  FFRDCs  to  determine 
during  comprehensive  reviews  whether  the  FFRDC  should  be 
discontinued  completely  or  whether  just  parts  of  the  FFRDC 
relationship  are  no  longer  justified. 

Manaaem**"**-  fiftinments.  The  Deputy  DDR&E,  the  Deputy  Assistant 
Secretary  of  Defense  (Strategic  and  Tactical  C^) ,  the  Navy,  the 
Air  Force,  and  the  Director,  Advanced  Research  Projects  Agency, 
concurred  with  the  need  to  discontinue  FFRDCs  that  were  not 
justified.  The  Deputy  Assistant  Secretary  of  Defense  (Strategic 
and  Tactical  C^)  and  the  Navy  stated  that  the  primary  sponsor,  as 
opposed  to  the  DDR&E,  should  be  responsible  for  discontinuing 
FFRDCs  that  are  not  adequately  justified. 

Audit  response.  Based  on  managements  comments,  we  revised 
our  recommendation  to  clarify  that  an  FFRDC  relationship  can 
be  restructured  in  part  or  discontinued  or  transferred  in 
its  entirety. 

We  ask  that  the  Deputy  DDR&E  provide  comments  in  response  to 
the  revised  recommendation  and  provide  details  on  procedures 
that  will  verify  that  sponsors  determine  during 
comprehensive  reviews  whether  FFRDC  relationships  should  be 
discontinued,  in  whole  or  part,  when  no  longer  justified. 
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d.  Revise  the  guidance  in  the  "implementation  of  the 
Federally  Funded  Research  and  Development  Center  Management  Plan" 
to  describe  the  purpose  and  extent  of  the  Director^  Defense 
Research  and  Engineering,  participation  in  the  conduct  of 
sponsors'  comprehensive  reviews. 

Added  naesftmmendation.  Based  on  oral  and  written  management 
comments,  we  added  the  above  recommendation  to  clarify  the  role 
of  the  DDR&E  in  sponsor's  comprehensive  reviews.  Accordingly,  we 
request  the  DDR&E  to  provide  comments  on  the  added  recommendation 
when  responding  to  the  report. 

2.  We  recommend  that  the  Director,  Defense  Research  and 
Engineering;  the  Deputy  Under  Secretary  of  Defense  for  Logistics; 
the  Assistant  Secretary  of  the  Navy  (Research,  Development,  and 
Acquisition) ;  the  Assistant  Secretary  of  the  Air  Force 
(Acquisition) ;  and  the  Director,  Defense  Supply  Se^ice- 
Washington  not  award  any  new  contracts  pending  the  completion  of 
an  acceptable  comprehensive  review  and  adequate  sole-source 
justification  for  the  applicable  FFRDC. 

Management  comment a.  The  Deputy  DDR&E  partially  concurred 
with  the  recommendation.  He  agreed  that  comprehensive  reviews 
should  be  performed  before  contract  awards  but  not  before 
exercising  annual  options  under  existing  contracts. 

The  Deputy  Assistant  Secretary  of  Defense  (Strategic  and  Tactical 
C^)  concurred  with  the  intent  of  the  recommendation  to  improve 
comprehensive  reviews  and  sole-source  justifications  but  believed 
that  the  recommendation  could  deny  DoD  a  source  of  support  that 
sponsors  find  invaluable. 

The  Army  nonconcurred  and  stated  the  comprehensive  review  for  the 
Arroyo  Center  was  adequate  and  that  requiring  another  review 
during  the  middle  of  the  contract  was  not  acceptable  business 
practice . 

The  Navy  nonconcurred  with  the  intent  of  the  recommendation  to 
halt  contract  modifications  and  thereby  cause  cessation  of  FFRDC 
operation  and  deprive  the  Navy  of  a  source  of  analytical 
research.  The  Navy  stated  it  agreed  with  the  underlying 
motivation  of  the  recommendation  to  improve  the  process  for 
conducting  comprehensive  reviews  and  improve  the  basis  for  award 
of  sole-source  contracts. 

The  Air  Force  concurred  with  the  need  to  perform  comprehensive 
reviews  before  award  of  new  contracts.  The  Air  Force  also  stated 
that  it  would  be  disruptive  and  delay  ongoing  programs  to  not 
exercise  options  and  modify  the  contract  pending  completion  of 
ongoing  comprehensive  reviews. 

The  Director,  Advanced  Research  Projects  Agency,  nonconcurred  and 
stated  the  basic  issue  of  whether  the  relationship  between  the 
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FFRDC  and  the  Government  requires  recompetition  every  5  years 
must  be  resolved  before  the  drastic  action  of  not  exercising 
options  to  current  contracts  cited  in  the  recommendation  occurs. 

Audit  response.  We  revised  our  recommendation  and  deleted 
that  portion  related  to  not  exercising  options  and 
modifications  to  the  contracts  before  completing  new 
comprehensive  reviews.  The  intent  of  the  recommendation  was 
to  encourage  the  conduct  of  new  comprehensive  reviews  to 
adequately  support  the  need  for  the  sole-source  contracts. 
We  have  no  desire  to  stop  ongoing  work.  The  responses  to 
Recommendation  l.a.  indicate  that  new  comprehensive  reviews 
were  completed,  ongoing,  or  planned  for  the  FFRDCs. 
Accordingly,  additional  comments  on  this  recommendation  are 
not  required. 

3.  We  recommend  that  the  Assistant  Secretary  of  the  Navy 
(Research,  Development,  and  Acquisition)  perform  cost  and  other 
analyses  to  determine  whether  replacing  all  or  a  portion  of  the 
Center  for  Naval  Analyses  field  analysts  with  less  costly 
in-house  personnel  is  feasible. 

Management  comments.  The  Navy  nonconcurred  with  the 

recommendation  and  stated  that  the  auditors'  sole  criterion  was 
cost  of  the  analysts.  The  Navy  stated  that  the  field  program  was 
one  of  the  primary  reasons  CNA  existed  as  an  FFRDC  and  that  the 
field  program  constituted  a  long-term  special  competency  and 
capability  that  can  not  be  met  as  effectively  by  existing  in- 
house  or  other  contractor  resources.  The  Navy  further  stated 
that  the  field  program  benefited  the  Navy  by  providing^  on-site 
analytical  support  and  that  this  field  experience  provides  the 
headquarters-based  research  program  a  degree  of  operational 
realism  that  is  unique  among  FFRDCs.  The  Navy  stated  that  the 
cost  of  duplicating  the  infrastructure  the  CNA  headquarters 
provides  in  support  of  the  field  program  was  not  considered  in 
the  audit.  Also,  personnel  with  the  needed  educational 
background  and  expertise  are  not  available  or  likely  to  be 
recruited  by  the  Navy.  The  Navy  concluded  that  significant 
economies  would  not  be  achieved  by  replacing  CNA  with  Navy 
resources  and,  in  today's  era  of  downsizing,  the  Navy  does  not 
have  the  billets  or  infrastructure  to  do  so. 

Audit  response.  We  revised  the  recommendation  to  request 
the  Navy  to  look  at  both  cost  and  any  other  factors  to 
determine  whether  replacing  all  or  portions  of  the  field 
analysts  was  practical.  We  agree  that  the  Navy  has  had  a 
long-term  relationship  with  CNA,  and  the  use  of  field 
analysts  does  benefit  the  Navy.  We  believe  the  Navy  should 
objectively  analyze  the  costs  and  benefits  of  continuing  the 
current  level  of  CNA  field  analysts.  The  Army  and  the 
Air  Force  do  not  use  field  analysts  from  their  FFRDCs.  The 
field  analysts  cost  the  Navy  about  $8.3  annually  (FY  1991 
costs)  and  the  planned  FY  1994  budget  for  CNA  is  to  decrease 
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by  about  10  percent  to  $45  million.  Accordingly,  the 
continued  use  of  CNA  field  analysts  at  their  present  level 
will  decrease  the  ability  of  CNA  to  perform  other  critical 
studies  for  the  Navy.  The  Navy  comments  indicate  the  Navy 
is  determined  to  maintain  the  CNA  field  analysts^  without 
considering  cost  or  efficiency.  The  intent  to  continue  this 
unique  Navy  arrangement  without  analyzing  alternatives  is 
not  justifiable.  Also,  neither  the  existence  of  nor  the 
need  for  extensive  headquarters  infrastructure  support  to 
the  field  program  analysts,  which  purportedly  would  have  to 
be  duplicated,  was  evident  during  the  audit. 

We  request  that  the  Navy  provide  additional  comments  on  the 
revised  recommendation  in  responding  to  the  final  report. 

RESPONSE  REQUIREMENTS  PER  RECOMMENDATION 


Response  should  cover 
Concur/  Proposed  Completion  Related 


Number 

Addressee 

Nonconcur 

Action 

Date 

Issues* 

l.a. 

DDR&E 

X 

l.b. 

DDR&E 

X 

X 

X 

IC 

l.c. 

DDR&E 

X 

X 

X 

IC 

l.d. 

DDR&E 

X 

X 

X 

3. 

ASN(RD&A) 

X 

X 

X 

M 

*  IC  = 

material  internal  control 

weakness; 

M  =  monetary 

benef its 
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-  Federal  Acquisition  Regulation  35.017, 
"Federally  Funded  Research  and 
Development  Centers" 

-  Office  of  Federal  Procurement  Policy 
Requests  for  Audit 

-  Summary  of  Prior  Audits  and  other 
Reviews 

->  Sole-Source  Renegotiation  Justifications 

-  Summary  of  Reviews  and  Justifications 

-  Details  on  Contracts  with  RAND 
Corporation  for  Project  AIR  FORCE, 

National  Defense  Research  Institute, 
and  Arroyo  Center 

-  Details  on  Contract  with  Institute  for 
Defense  Analyses 

-  Details  on  Contract  with  Center  for 
Naval  Analyses 

-  Details  on  Contract  with  Logistics 
Management  Institute 

-  Details  on  Contract  with  Aerospace 
Corporation 

-  Details  on  Contracts  with  MITRE 
Corporation  C^I  Division 

-  Details  on  Contract  with  Lincoln 
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See  footnotes  at  end  of  appendix. 
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EXECUTIVE  oi^nCE  OF  THE  PRESIDENT 
OFFICE  OF  MANAGEMENT  AND  BUDGET 
WASHINOTQM.  O.C  lOM 

orricx  or  rcoCRAL 

ritOCURCMKfiT 
rocicv 

OFPP  POLICY  LETTER  ll-l 
TO  THE  HEADS  OF  EXECUTIVE  DEPARTMENTS  AND  ESTABLISHMENTS 
SUBJECTS  Federally  Funded  Research  and  Development  Centers 


APR4 


1.  Purpose.  This  policy  letter  establishes  Government-wide  policies  for  the 
establishment,  use,  periodic  review,  and  termination  of  the  sponsorship  of 
Federally  Fun^  Research  and  Development  Centers  (FFRDCs). 

2.  Supersession.  Memorandum  from  the  Chairman  to  the  Members  of  the  Federal 
Council  for  Science  and  Technology,  dated  November  1,  19(7,  which  set  forth 
criteria  for  identification  6f  FFRDCs  and  the  reqt^ement  for  a  master 
Government  listing  of  these  centers,  b  superseded  by  thb  policy  letter. 

3.  Au’^rltv.  Thb  policy  letter  b  being  Issued  pursuant  to  Sections  ((a),  (CdXO 
and  ((dXS)  of  the  Office  of  Federal  Procurement  Policy  Act,  as.  amend^  #1 
U.S.C  *03  (a),  (dXI)  and  (dXS)^  which  empower  the  Adminbtrator  of  OFPP  to 
prescribe  Government-wide  procurement  policies  and  to  comptete  action  on  the 
recommendations  of  the  Commission  on  Government  Procurement. 

4.  Badrground.  The  Departments  of  Energy,  Defense,  Health  and  Human 
^rvices,  the  National  Aeronautics  and  Space  Administration  and  the  National 
Science  Foundation  currently  ^onsor  a  total  of  3*  FFRDCs.  Non-sponsoring 
departments  and, agencies  also  utilize  these  FFRDCs.  Federal  ftmdlng  of  FFRDCb 
currently  exceeds  •  billion  dollars  per  year. 

In  1967,  a  Government-wide  policy  for  the  Identification  aitd  maintenance  of  a 
master  Ibting  of  these  FFRDCs  was  issued  (reference  paragraph  2  •  Supersession). 
In  1972,  the  Commission  on  Government  Procurement  recommended  that  the 
Federal  Government  keep  open  the  option  to  organize  and  use  FFRDCs  to  satisfy 
needs  that  cannot  be  satbfled  effectively  by  other  organizational  resources.  The 
Commbslon  also  recommended  that  agency  hean  periodically  review  the 
continuing  need  for  exbtlM  FFRDCs  and  approve  any  proposal  for  new  FFRDCs, 
with  specific  attention  paid  to  the. method  of  ultimate  termination  of  censorship. 
Thb  policy  letter  b  based  on  the  executive  branch  consideration  of  the 
Commbsion's  recommendations. 

3.  Definitions 

a.  Primary  Sponsor  —  The  executive  agency  which  manages,  adminbters  or 
monitors  overall  use  of  the  FFRDC. 
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b.  Soonjor  meani  w  executive  agency  which  fun^s  and  nnonitort  ipeciflC 
work  of  ecmtinuing  nature  with  an  FFRDC  and  is  party  to  a  sponsoring 
agreement.  Multiple  sponsorship  of  an  FFRDC  is  possible  so  long  as  one  agency 
agrees  to  act  as  the  primary  sponsor  for  administrative  purposes. 

c.  Federally  Funded  Research  and  Development  Center  (FFRDC). 

(1)  FFRDCs  do  not  have  a  prescribed  organizational  structure.  They  can 
range  from  the  traditional  contractor-owned/contractor-operated  or  Cov- 
ernment-owned/contractor-operated  (COCO)  organizational  structures  to 
various  degrees  of  contractor/Govemment  control  and  ownership.  In 
general,  however,  all  of  the  following  criteria  should  be  met  before  an 
activity  Is  Identified  as  an  FFRDCr 

(a)  Performs,  analyzes,  integrates,  supports  (non-financial)  and/or 
manages  basic  research,  applied  research,  and/or  develc^ment. 
(Activities  primarily  engaged  in  routine  quality  control  and 
testing,  routine  service  activities,  production,  mapping  and 
surveys,  and  Information  dissemination,  even  though  otherwise 
meetli^  die  requirements  of  paragraph  5^  are  specifically 
excluded  from  FFRDC  designation). 

(b)  Performance  of  the  functions  in  l.c.(lXa)  is  either  upon  the 
direct  request  of  the  Government  or  under  a  broad  dtarter  from 
the  Government,  but  in  either  case  the  remits  are  directly 
monitored  by  the  Government.  However,  the  monitoring  shall 
not  be  such  as  to  create  a  personal  services  relatlondtlp,  or  to 
cause  disruptions  that  are  detrimental  to  the  productivity 
and/or  quality  of  the  FFRDCs  work. 

(c)  The  majority  of  dw  activity^  financial  support  (70%  or  more)  is 
received  from  the  Government  with  a  single  agency  uwally 
predominating  in  that  financial  support. 

(d)  In  general,  most  or  all  of  the  facilities  are  owned  by  the 
Government  or  funded,  under  contract,  by  the  Government. 

(e)  The  activity  is  operated,  managed  and/or  administered  by 
either  a  university  or  consortium  of  universities,  other  non- 
profit  organization  or  industrial  firm  as  an  autonomous 
organization  or  as  an  identifiable  separate  operating  unit  of  a 
parent  organization. 

(f)  A  long  term  relationship  evidenced  by  specific  agreement  exisU 
or  is  expected  to  exist  between  the  operator,  manager,  or 
administrator  of  the  activity  and  its  primary  sponsor. 
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(2)  kt  «ddition  to  the  above  aitcria,  the  relationship  between  the 

activity  and  the  'Covcrnmcnt  should  exhibit  the  following 

characteristics  In  order  to  qualify  for  FFROC  identificatiora 

(a)  The  activity  (organization  and/or  facilities)  b  brought  Into 
existence  at  the  initiative  of  a  Government  agency  or  bureau  to 
meet  some  special  research  or  development  need  which,  at  the 
time,  cannot  be  met  as  effectively  by  existing  iis-housc  or 
contractor  resources. 

(b)  Work  from  other  than  a  sponsoring  agency  Is  undertaken  only  to 
the  extent  permitted  by  the  sponsoring  agency  and  in 
accordance  with  the  procedures  of  the  sponsoring  agency. 

e)  The  activity,  whether  the  operator  of  Its  own  or  a  Government- 
owned  facility,  has  access,  beyond  that  which  is  common  to  the 
normal  contractual  relationship,  to  Government  and/or  supplier 
dau,  employees,  and  facilities  needed  to  discharge  Its 
responsibiiities  efficiently  and  effectively,  whether  the  data  Is 
sensitive/proprietary  or  not. 

d)  The  primary  sponsor  undertakes  the  responsibility  to  *<*we  a 
reasonable  continuity  In  the  level  of  support  to  the  activity 
consistent  with  the  agency's  need  for  the  activity  and  the  terms 
of  the  sponsoring  agreement. ' 

e)  The  activity  is  required  to  conduct  its  buslrtess  in  a  responsible 
manner  befitting  fts  special  relationship  with  the  Government, 
to  operate  In  the  public  btterest  free  from  erganizatiortal 
conflict  of  Interest,  and  to  disclose  its  affairs  (as  an  FFRDC)  to 
the  primary  sponsor. 

S.  Policy. 

a.  CeneraL  Agencies  will  rely,  to  the  extent  practicable,  on  existing  in- 
house  and  contractor  sources  for  satisfying  their  special  research  or  developmmt 
needs  consistent  with  established  procedures  under  The  Economy  Act  of  IT32  (31 
use  1333),  other  statutory  authority  or  procurement/assistance  regulations.  A 
thqrou^  assessment  of  existing  alternative  souces  for  meeting  these  needs  is 
especially  important  prior  to  establishing  an  FFRDC.  This  Policy  Letter  does  imt 
apply  to  the  performance  of  commercial  activities.*  Performance  of  commercial 
activities  is  governed  by  0MB  Ckcular  No.  A-7<. 
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b*  Establishment  ef  in  FFRDC*  In  establishing  an  FFROC9  the  sponsoring 
agency  shall  ensure  thats 

(0  Existing  alternstlve  sources  for  satisfying  agency  requirements 
cannot  effectively  meet  the  special  research  or  development  needs 

(2)  At  least  three  notices  are  placed  ever  a  90<day  period  in  the 
Commerce  Business  Dally  and  The  Federal  Retister  indicating  the 
agency's  intention  to  sponsor  an  FFRDC  and  the  scope  and  nature  of 
the  effort  to  be  performed  by  the  FFRDC. 

(3)  There  is  sufficient  Government  expertise  available  to  adequately  and 
objectively  evaluate  the  work  to  be  performed  by  the  FFRDC. 

(4)  Controls  are  estabUshed  to  ensure  that  the  cosu  ef  the  services  being 
provided  to  the  Government  are  reasonable. 

(5)  The  responsibility  for  capitalization  of  the  FFRDC  has  been  defined 
in  such  a  manner  that  ownership  of  assets  may  be  readily  and 
equitably  determined  upon  termination  of  the  FFRDC  relationsh^ 
with  its  sponsoKs). 

(6)  The  purpose,  mission  and  general  scope  of  effort  of  the  FFRDC  is 
stated  clearly  enough  to  enable  differentiation  between  work  which 
should  be  performed  by  the  FFRDC  and  that  which  should  be 
performed  by  a  non-FFROC. 

c.  SponsorihK  Agreements.  When  FFRDCs  are  established,  long-term 
Government  relationships  are  encouraged  in  order  to  provide  the  continuity  that 
will  attract  high  quality  personnel  to  Ae  FFRDC  This  relationship  should  be  ef  a 
type  to  encourage  the  FFRDC  to  maintain  currency  in  its  fkldfs)  of  expertise, 
maintain  its  objectivity  and  independence,  preserve  its  familiarity  with  the  needs 
of  its  sponsoKs),  and  provide  a  qqkk  response  capability.  A  contract  is  the 
generally  preferred  instrument  under  which  an  FFRDC  accomplishes  effort  for  its 
sponsoKs).  However,  there  may  be  instances  where  other  legal  instruments  may  be 
appropriate.  A  written  agreement  of  sponsorship  between  tfte  FFRDC  and  its 
sponsor  or  primary  sponsor  where  more  than  one  sponsor  is  involved  may  be  used  In 
addition  to  the  contract  or  other  legal  instrument  under  which  an  FFRDC 
accomplishes  effort.  The  specific  content  of  a  sponsoring  agreement  will  vary 
depending  on  the  situation.  However,  there  are  certtin  areas  common  to  aU 
situations  that  must  be  addressed.  The  following  requirements  must  be  addressed 
In  either  a  contract,  a  sponsoring  agreement  or  sponsoring  agency's  policies  and 
procedures. 
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U)  Mandttonr  ftc^kcfflcnU  } 

U)  A  ^OnettlM  •!  «M  purpow  for  which  ttit  FFRDC  li  b«ln|  brogcht  Into 
hclnc  oloM  with  •  ^icrlptloo  of  lu  mlulon*  fcnerol  acopt  of  effort 
envblooed  to  bo  perform^  oM  the  role  die  FFRDC  h  to  hove  In 
occompDihment  of  the  sponiorint  of mbtlon.  IhU  tfellneetten  mutt 
be  cenilftcnt  with  tfie  definition  of  on  FFRDC  oet  forth  bi  porofroph 
fx<lXa)  and  will  be  ouffklentb  descrbtlve  to  tfut  work  to  be  Mrformed 
by  the  FFRDC  can  be  deternuned  to  be  within  the  purpoMi  mftslon  and 
teneral  tcope  of  effort  for  whldt  ttte  FFRDC  wu  ettabliihed  and 
dIfferentUtM  from  work  which  aheuM  be  performed  by  a  non*FFRDC 
This  dellneatlen  shall  constitute  the  base  aialnst  which  chances  In  an 
cxlstlni  FFRDCb  purposoi  mbslon  or  general  scope  of  effort  will  be 
measured. 

(b)  Frovlslent  for  the  orderly  termination  or  nonrcnewal  of  the  agreement, 
disposal  of  assets  and  settlement  of  liabilities.  The  term  of  the 
sponsorsing  agreement  will  not  esceed  five  years  but  can  be  renewed,  as  a 
result  of  periodic  review.  In  net  to  exceed  five  year  Increments. 

(c)  A  prohibition  against  the  FFRDC  competing  with  any  non-FFRDC  concern 
In  response  to  a  Federal  agency  formal  Request  For  Fropoul  for  ether 
than  the  operation  of  an  FFRDC.  This  prohibition  Is  not  required  to  be 
applied  to  any  parent  organization  or  other  subsidiary  of  the  parent 
organization  In  Its  non-FFRDC  operations.  However,  ^soring  agencies 

may  expand  this  prohibition  u  they  determine  neces^  and  appropriate. 

« 

0  « 

(d)  A  deOneation  of  whether  or  net  the  FFRDC  may  accept  work  from  other 
'than  the  ^onsoKs).  If  non-monsor  work  can  be  accepted^  a  delineation 
of  ttte  procedures  to  be  followed  along  with  any  Dmltations  as  to  the 
clients  (other  Federal  agencies,  Sute  or  local  governments,  non-profit  or 
profit  organizatlont,  etc.)  from  which  work  may  be  accepted.  Limitations 
and  procedures  with  respect  to  responding  to  requests  for  Information  as 
to  an  FFROCs  capabilities  or  qualifications  are  Inherently  a  part  of  the 
Vork  for  others*  question  and  will  be  addresMd  by  tfic  ^soring  agency. 

C2)  Other  Requirements  As  Appropriate 

Ca)  When  cost  type  contracts  arc  used,  the  spontorfs)  should  Identify  any 
cost  elements  which  will  require  advance  agreement.  Such  Items 
atay  be,  but  are  not  necessarily  limited  tea  salary  structure, 
depreclatloiv  various  Indirect  costt  such  as  Independent  resesr^  and 
development  ar  ethers  as  determined  appropriate  by  the  spensoKi). 

(b)  Where  fees  are  determined  Iw  tee  to  be  appropriate, 

consMeretlons  which  will  Mtect  teeir  negotiation  aheuM  be 
Identified.  Such  considerations  may  bo,  but  arc  not  necessarily 
limited  to,  weighted  guidelines,  risks,  use  of  Covemment  himished 
property  vtd  facilities,  needs  or  others  as  determined  appropriate  by 
.tec  iponsoKs). 
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(c)  Other  provisions  as  determined  approprbte  by  the  ^onsoKs). 


134 


d.  Chmnrint  the  Basle  Scope  of  an  ExiitlnK  FFRDCs  SponsorlnK  Agreement. 
In  chanelnc  the  purpose,  mission  and  general  scope  of  effort  to  be  perf^rned  or 
role  of  an  exlstlnc  FFRDC  as  set  forth  in  its  qionsorlnt  agreement  (see  <x.(lXa}). 
tfte  sponsorlnt  agency  shall  make  such  changes  consistent  with  its  *^lwto^ 
authority  and  the  requirements  for  estshllshing  a  new  FFRDC  as  set  forth  In 
paragraph  (.b. 


a.  use  of  the  FFRDC  bv  the  Sponsor  or  Primary  Stwnsor  In  the  Case  of 
Multiple  Arencv  Sponsorship.  The  prlinary, 

multiple  spmsorship,  wUI  ensure  that  ail  work  It  places  with  Its  FFRDC(t)  is  within 
the  purpole;  mission,  and  general  scope  of  effort  of  the  FFRDC  (para|raph  <.c.) 
and  to  accordance  with  this  Policy  Utter.  This  Includes  work  a  sponsoring  age^ 
agrees  to  accept  from  a  non-sponsori/w  Federal  agency  under  the  provisions  of  too 
Uonomy  Act  of  IJ32  (31  USC  153»  or  other  statutory  authority.  Sponsortog 
agencies  must  comply  widi  applicable  procurement  or  assistance  statute^  policies 
and  regulations  for  non-competitive  .actions  before  placing  work  whirt  b  wtside 
the  scope  of  the  sponsor's  contractual  or  ^onsortog  agreement  with  an  FFRDC. 

f.  of  an  Existing  FFRDC  bv  a  Non-Sponsoring  Federal 

sponsoring  Federal  agencies  may  use  an-FFRDC  only  11  the  terms  of  the  FFRDC  s 
sponsoring  agreement  or  contract  permit  work  from  other  than  a  sponsoring 
agency.  Where  use  by  a  non-sponsor  is  permitted  bj^e  Sponsoring  Agreement,  the 
work  must  require  the  special  relationship  of  an^RDC  as  defined  to  paragr^ 
5.C.  and  either  be  treated  as  a  direct  procurement  (action)  or  oroceMed  under  The 
Economy  Act  of  »32  (31  USC  1333)  or  other  statutory  authority.  Work  processed 
under  The  Economy  Act  of  1932  (31  USC  1333)  or  other  statutory  authority  njurt 
clearly  fall  within  the  purpose,  mission  and  general  scoU  of  effort  established^ 
the  SMnsoring  agency  for  the  FFRDC  (paragraph  Proce«if«  under  fte 

Econ^y  Act  or  other  sUtutory  authority  b  siiject  to  agreenwnt  by  the  receiving 
agency.  Non-sponsoring  agencies  must  fully  comply  with  prowement  or 
a»l$tance  sututes,  policies  and  regubtions  for  non-competitlve  acti<ms  *o 
pbcing  work  directly  with  a  specific  FFRDC  The  FFRDC 
mocedures  establbhed  by  the  sponsortog  agency  (paragraph  Cx.(lXd))  before 
accepting  work  from  a  non-sponsorli\f  Federal  agency* 

g.  nf  an  Eslstint  FFRDC  bv  Other  Than  a  Federal  Agency.  . 
other  than  a  Federal  agemS'  may  be  accepted  only  to  the  extent  permltt^  by  the 
sponsortog  agency.  The  FFRDC  must  comply  with  the  procedures  established  by 
the  ^onsortog  agency  paragraph  d.c.(lXd))  before  accepting  work  from  other  than 
a  Federal  agency. 
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h.  Cenmltini  Services.  Agencies  ^onsoring  FFROC  work  which  constitutes 
consulting  service^  as  defined  by  0MB  Circular  No.  A>120,  will  comply  with  the 
provisions  of  that  Circular. 

1.  Production/Manufacturing.  FFRDCs  will  not  be  asked  to  perform 
quantity  production  and  manufacturing  work  unless  authorized  by  legislation.  Such 
activities  as  breadboarding,  modeling  or  'other  tasks  inherent  to  RAO  are 
permissible. 

1*  Periodic  Review.  Prior  to  renewal  of  a  sponsoring  agreement,  agencies 
shall  conduct  a  comprehensive  review  of  their  use  and  need  for  each  FFROC  that 
they  sponsor.  Where  multiple  agency  sponsorship  exists  this  review  will  be  a 
coordinated  interagency  effort.  When  the  funding  for  an  FFROC  is  a  specific  line 
item  within  the  sponsoring  agency's  budget,  the  comprehensive  review  may  be  done 
in  conjunction  with  the  budget  process  or  the  review  may  be  done  separately.  The 
sponsoring  agencyfs)  shall  apprise  other  agencies  who  use  the  FFROC  of  the 
scheduled  review  and  afford  d^m  an  opportunity  to  assume  sponsorship  in  the 
event  the  current  sponsorship  is  determined  no  longer  appropriate.  Final  approval 
to  continue  or  terminate  an  agency's  q>onsorship  arrangement  with  a  given  FFROC 
as  a  result  of  this  review  shall  rest  with  the  head  of  that  sponsoring  agency.  The 
results  of  this  review  will  be  formally  documented.  The  periodic  review  should 
include: 

(1)  An  examination  of  the  agency's  special  technical  needs  and  mission 
requirements  to  determine  if  and  at  what  level  they  continue  to 
exist. 

(2)  Consideration  of  alternative  sources  to  meet  the  agency's  needs. 
Su^  consideration  will  include  compliance  with  the  Notice  and 
Publication  requirements  of  P.L.  9S-72  (19  USC  d37(e)}  prior  to 
renewal  of  the  contract  or  Sponsoring  Agreement  unless  otherwise 
exempted. 

O)  An  assessment  of  the  efficiency  and  effectiveness  of  the  FFRDC  in 
meeting  the  agency's  needs. 

~(«)  An  assessment  of  the  adequacy  of  the  FFRE>C  management  In 
assuring  a  cost  effective  operation. 

(9)  A  determination  diat  the  guidelines  of  section  6  are  being  satisfied. 

k.  Termination  or  nonrenewal  of  an  FFRDC  Relationdilo.  When  a  sponsor's 
need  for  the  FFRDC  no  longer  exists,  the  sponsorship  may  be  uansferred  to  one  or 
more  Government  agencies,  if  appropriate^  justified.  Otherwise  it  shall  be  phased 
out,  the  assets  disposed  of  and  all  liabilities  settled  as  provided  by  the  terms  and 
conditions  of  the  sponsoring  agreement. 
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7.  Action  Reouirements. 

«.  Not  later  than  September  30,  19S*,  each  agency  currently  iponsoring  an 
FFRDC  will  review  the  terms  of  its  existing  agreements  with  the  FFRDCs  for 
compliance  with  this  policy  letter.  Where  existing  agreements  do  not  comply  wl^ 
this  policy  letter  the  primary  ^onsor  will  develop  a  schedule  to  brii^  the 
agreements  into  compliance  not  later  than  the  next  contract  renewal  or  five  years 
from  the  effective  date  of  this  ^licy,  letter,  whichever  comes  first. 

b.  Where  the  review  required  by  7.a.  reveab  that  a  clear  statement  of  the 
purpose,  mission  and  general  scope  of  effort,  as  described  in  paragraph  6.b.(d)  and 
6.c.(lXa),  does  not  exist,  the  sponsoring  agency  shall  ensure  such  a  statement  Is 
developed  not  later  than  September  30, 19S4. 

c.  The  primary  sponsor  will  notify  the  Office  of  Science  and  Technology 
Policy  prior  to  designating  any  new  organization  as  an  FFRDC  (paragraph  d.b.), 
changing  the  basic  scope  of  effort  of  an  existing  FFRDC  (paragraph  g.d.)  or 
changing. the  sutus  of  an  existing  FFRDC  (paragraph  g.l(.). 

d.  The  National  Science  Foundation  will  maintain  a  ntaster  Government  list 
of  FFRDCs  based  upon  the  definition  in  this  Policy  Letter. 

e.  FFRDCs  will  be  identified  by  their  primary  sponsors  who  will  provide 
information,  inchidl/^  funding  data,  on  the  tyi«  of  R&D  being  performed  by  the 
FFRDCs  to  the  National  Science  Foundation  upon  their  request  for  such 
information. 

f.  Each  agency  head  is  responsible  for  ensuring  that  the  provisions  of  this 
policy  are  followed. 

g.  Effective  Date.  The  Policy  Letter  is  effective  «0  days  after  publication  fat  the 
Federal  ReeisteryT 

9.  implementation.  Aspecu  of  this  policy  letter  requiring  Implementation  wUl  be 
covered  by  the  Department  of  Defense,  the  General  Services  Admlnbtration  and 
the  National  Aeronautics  and  Space  Admlnbtration  in  the  Federal  Acquisition 
Regulation  not  bter  than  ISO  days  from  the  date  of  thb  policy  fetter. 
Implementation  will  be  written  so  as  to  be  compatible  with  the  requirements,  as  of 
the  date  of  thb  policy  fetter,  of  FAR  17.6  "Management  and  Operating  Contracts" 
when  the  arrangoment  with  an  FFRDC  constitutes  a  management  and  operating 
contract. 

10.  Information  ConUct.  All  questions  or  inquiries  about  thb  policy  fetter  should 
be  submitted  to  the  6ffice  of  Management  and  Budget,  Office  of  Federal 
Procurement  Policy,  telephone  (202)  393-6610. 

11.  Sunset  Review  Date.  Thb  policy  letter  will  be  reviewed  no  later  than  six 
years  after  its  effective  date  for  extension,  modification,  or  rescission. 


DonaVE.  Sowle 
Adrpmistrator 
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APPENDIX  C  -  FEDERAL  AC0DI8ITI0N  REGULATION  35.017, 
"PEDERALLY  PUMPED  RESEARCH  AND  DEVELOPMENT  CENTERS** 


35jM7 _ _ 

35J17  Federally  Funded  Rcscarcb  and  DevelopmeDt 
Centera* 

(a)  foiky. 

(1)  This  section  sett  fonh  FedenJ  poliqr  regarding 
the  esublishmenu  use,  review,  and  termination  of 
Federally  Funded  Research  and  l>evelopment  Centers 
(FFRDCs)  and  related  sponsoring  agreements. 

(2)  An  FFRDC  meets  some  special  long-term 
reaettcb  or  developncDt  need  which  cannot  be  met  at 
cfibcaveJy  by  existing  in-house  or  eonoacsor  resources. 
FFRDCs  enable  agcscies  to  use  private  sector  resourecs 
10  accomplish  tasks  that  are  imegral  to  the  missioA  and 
opertiioo  of  the  sponsoring  agency.  An  FFRDC.  in 
order  to  discharge  its  rcsponsibiliries  to  the  sponsoring 
agency,  has  access,  beyond  that  which  is  common  lo  the 
aonnal  cootractuai  relationship,  to  Government  and  sup¬ 
plier  data,  iDcluding  sensitive  and  pR)prietary  data,  and 
to  employees  and  CKilides.  •  The  FFRDC  is  required  to 
conduct  its  business  in  a  manner  befitting  its  special 
reladcxiship  with  the  Government,  to  operate  in  the  pub¬ 
lic  iaaerest  with  objeoivicy  and  independence,  to  be  free 
fit»  organizabonaJ  conflkns  of  interest,  and  to  have  full 
dbdosure  of  its  afTaire  to  the  sportsoring  agency.  It  is 
not  the  Govemmenfs  intent  that «  FFRDC  use  its  priv¬ 
ileged  information  or  access  to  fhcilities  lo  compete 
with  the  private  sector.  However,  an  FFRDC  may  per¬ 
form  work  for  other  than  the  sponsoring  agency  under 
the  Economy  Act,  or  other  a^^Iicable  legislation,  when 
the  work  is  not  otherwise  available  from  the  private  sec¬ 
tor. 

(3)  FFRDCs  are  opemed,  managed,  andAv  adminis- 
tted  by  either  a  university  or  consortium  of  universiues. 
other  oot-fcY-profit  or  nonpn^nt  organization,  or  as  indus¬ 
trial  firm,  as  an  autononxxis  orgaruzaiioo  or  as  an  identi¬ 
fiable  separate  operating  unit  of  a  parent  orgamzadon. 

(4)  Long-tenn  relationships  between  the  Govemmem 
and  FFRDCs  are  encouraged  in  order  to  provide  the 
continuity  that  wiS  aoraa  high-quality  penonnel  lo  the 
FFRDC.  This  relationship  should  be  of  a  type  to 
encourage  the  FFRDC  to  maintain  currency  in  its 
fieldfs)  of  expertise,  maintain  its  objectivity  and  inde¬ 
pendence,  preserve  its  famOiarity  with  the  needs  of  its 
spOQsor(s),  and  provide  a  quick  response  capabilicy. 
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(b)  Dr/Snitioru. 

*^<loQsponsor.*  as  used  in  this  section,  means  any  other 
organization,  in  or  outside  of  (he  Federal  Government, 
which  fuids  specific  work  to  be  performed  by  (he  FFRDC 
nd  is  not  a  party  to  the  sponsoring  agreemenL 

*Trimary  sponsor,*  as  used  in  this  section,  means  the 
lead  agency  responsible  for  managing,  administering,  or 
monitoring  ovcfall  use  of  the  FFRDC  under  a  multiple 
kpoosoeship  agreemem. 

"Special  competency,"  as  used  in  this  section,  means  a 
special  or  unique  capability,  including  qualitative  aspects, 
(kveloped  incidenul  to  the  primary  functions  of  the 
FFRDC  to  meet  some  ^>edal  need. 

"Sponsor"  means  the  executive  agency  which  manages, 
adminisien,  monitors,  funds,  and  is  responsible  for  the 
overall  use  of  an  FFRDC  Multiple  agency  sponsorship  is 
possible  as  long  as  one  agency  agrees  to  act  as  the 
"primary  sponsor."  In  the  event  of  multiple  sponsors, 
"sponsor^^ecs  to  the  primary  sponsor. 
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PART  35— RESEARCH  AND  DEVELOPMENT  CONTRACTING 


SpoQsoring  ftfrceiDCBti* 

(b)  In  order  to  ficiliute  i  lonf-ierm  reUtiooship 
between  the  Government  and  an  FFRDC,  establish  the 

FnUXTs  mission.  aiKl  ensuit  a  periodic  leevalid^ 

FFRDC.  a  written  agreement  of  sponsorship  between  the 
Covenunem  and  the  FFRDC  shall  be  prepared  when  the 
FFRDC  is  established.  The  sponsoring  agreement  may 
take  various  forms;  it  may  be  included  in  a  contract 
between  the  Government  and  the  FTODC,  or  in  another 
legal  instrument  under  which  an  M’kDC  accomplishes 
effort,  or  it  may  be  in  a  separate  written  agieemeuL 
Notwithstanding  its  form,  the  sponsoring  agreement  shall 
be  clearly  designated  as  such  by  the  sponsor. 

(b)  While  the  specific  content  of  any  sponsoring  agree¬ 
ment  will  vary  depending  on  the  situation,  the  agreement 
shall  contain,  as  a  minimum,  the  requirements  of  paragraph 

(c)  of  this  subsection.  The  requirements  for,  and  the  con¬ 
tents  of,  sponsoring  agreements  may  be  as  further  specifted 
in  sponsoring  agencies’  policies  and  procedures. 

(c)  As  a  minimum,  the  following  requirements  must  be 
addressed  in  either  a  sponsoring  agreement  or  ^lonsoring 
agencies'  policies  and  procedures: 

(1)  A  statement  of  the  purpose  and  mission  of  the 
FFRDC. 

(2)  Provisions  for  the  orderly  termination  or  nonre¬ 
newal  of  the  agreement,  disposal  of  assets,  and  setUe- 
meni  of  liabilities  The  responsibility  for  capitalization 
of  an  FFRDC  must  be  defined  in  such  a  manner  that 
ownership  of  assets  may  be  readily  and  equitably  deter¬ 
mined  upon  termination  of  the  FFRDCs  relationship 
with  its  sponsor(s). 

(3)  A  provision  for  the  identification  of  retained 
earnings  (reserves)  and  the  development  of  a  plan  for 
their  use  and  disposition. 

(4)  A  prohibition  against  the  FFRDC  competing 
with  any  non-FFRDC  concern  in  response  to  a  Federal 
agency  request  for  proposal  for  other  than  the  operation 
of  an  FFRDC.  This  prohibition  is  not  required  to  be 
applied  to  any  parent  organization  or  other  subsidiary  of 
the  parent  organization  in  its  non-FFRDC  operations. 
Requests  for  information,  qualificaxiohs  or  capabilities 
can  be  answered  unless  otherwise  restricted  by  the  spon¬ 
sor. 

(5)  A  delineation  of  whether  or  not  the  FFRDC  may 
accept  work  from  other  than  the  sponsor(s).  If  nonspon¬ 
sor  work  can  be  accepted,  i  delineation  of  the  proce¬ 
dures  to  be  followed,  along  with  any  limitations  as  to 
the  nonsponsors  from  which  work  can  be  accepted 
(other  Federal  agencies.  Suie  or  local  govemmenu, 
nonprofit  or  profit  organizations,  etc.). 

(d)  The  sponsoring  agreement  or  sponsoring  agencies’ 
policies  and  procedures  may  also  contain,  as  appropriate, 
other  provisions,  such  as  identification  of — 

(1)  Any  cost  elements  which  will  require  advance 
agreement  if  cost-type  contracts  are  used;  and _ 


(2)  Considentions  which  will  affect  negotiation  of 

fees  where  payment  of  fees  ti  determined  by  the  spon- 

aoKs)  to  be  appropriate. 

(c)  The  lenn  of  the  agreement  will  not  exceed  5  years, 
but  can  be  renewed,  as  a  result  of  periodic  review,  in  incre¬ 
ments  not  u  exceed  S  yean. 

354)17-2  Establishiog  or  cbaDgiiig  an  FFRDC. 

lb  establish  an  FFRDC,  or  change  its  basic  purpose  and 
mission,  the  sponsor  shall  ensure  the  following: 

(a)  Existing  altemative  sources  for  satisfying  agency 
requirements  cannot  effectively  meet  the  special  research 
or  development  needs. 

(b)  notices  required  for  publication  (see  S. 205(b)) 
are  placed  as  required. 

(c)  There  is  sufficieni  Government  expenise  available 
to  adequately  and  objectively  evaluate  the  work  to  be  per¬ 
formed  by  the  FFRDC. 

(d)  The  Executive  Office  of  the  President,  Office  of 
Science  and  Technology  Policy,  Washington,  DC  20506,  is 
nodfied. 

(e)  Conirolsareestablishedioensurethatthecostsofthe 
services  being  provided  to  the  Government  are  reasonable. 

({)  The  basic  purpose  and  mission  of  the  FFRDC  is  stat¬ 
ed  clearly  enough  to  enable  diffcreniiaiion  between  work 
which  should  be  performed  by  the  FFRDC  and  that  which 
should  be  performed  by  non-FFRDCs. 

(g)  A  reasonable  continuity  in  the  level  of  support  to  the 
FFRDC  is  maintained,  consistent  with  the  agency's  need 
for  the  FFRDC  and  the  terms  of  the  sponsoring  agr^ent 

(h)  The  FFRDC  is  operated,  managed,  or  administered 
by  an  autonomous  organization  or  as  an  identifiably  sepa¬ 
rate  operating  unit  of  a  parent  organization,  and  is  required 
to  operate  in  the  public  interest,  free  from  organizational 
conflict  of  interest,  and  to  disclose  iu  affairs  (as  an 
FFRDC)  10  the  primary  sponsor. 

(i)  0MB  Circular  A- 120  is  complied  with  when 
applicable,  artd  quantity  production  or  manufacturing  is  not 
performed  unless  authorized  by  legislation. 

0  Approval  is  received  from  the  head  of  the  sponsoring 
agency. 

354)17.3  Using  an  FFRDC. 

(a)  All  work  pUced  with  the  FFRDC  must  be  within  the 
purpose,  mission,  general  scope  of  effort,  or  special  compe¬ 
tency  of  the  FFRDC. 

(b)  Where  the  use  of  the  FFRDC  by  a  nonsponsor  is 

penniued  by  the  sponsor,  the  sponsor  shall  be  responsible 
for  compliance  with  paragraph  (a)  of  this  subsection.  The 
nonsponsoring  agency  is  responsible  for  making  the  deter¬ 
mination  required  by  17.502  and  providing  the  documenta¬ 
tion  required  by  17.504(e).  Wh«t  pcrmiiicd  by  the  spon¬ 
sor,  a  Federal  agency  may  contract  directly  with  the 
FFRDC  in  which  case  that  Federal  agency  is  responsible 
for  compliance  with  Pan  6. _  35-7 
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35U)17«4  Reviewing  FFRDC'ft. 

(a)  The  sponsor,  prior  to  extending  the  eontmet  or 
agreement  with  an  FFROC,  shall  ooodua  a  comprehenave 
review  of  the  use  and  need  for  the  FFRDC.  The  review 
will  be  coordinated  with  any  oo-qxnsors  and  may  be  per¬ 
formed  in  conjunction  with  the  budget  process.  If  the 
sponsor  determines  that  its  sponsorship  is  no  longer  appro¬ 
priate.  it  shill  apprise  ocher  agencies  which  use  the  FHUXJ 
of  the  dcterminaiion  and  afford  them  an  opportunity  to 
assume  sponsorship. 

(b)  Approval  to  continue  or  teminate  the  sponsorship 
shall  rest  with  the  head  of  the  sponsoring  agency.  This 
determmation  shall  be  based  upon  the  results  of  the  review 
conducted  in  accordance  with  paragraph  (c)  of  this  substc- 
lion. 

(c)  An  FFRDC  review  should  include  the  following; 

(1)  An  examination  of  the  sponsor's  special  technical 
nee^  and  mission  requirements  that  are  performed  by 
the  FFRDC  to  determine  if  and  at  what  level  they  con¬ 
tinue  to  exisL 

(2)  Consideration  of  alternative  sources  to  meet  the 
sponsor's  needs. 

(3)  An  assessment  of  the  efficiency  and  effective¬ 
ness  of  the  FFRDC  in  meeting  the  sponsor's  needs, 
including  the  FFRDCs  ability  to  maintain  its  objectivi¬ 
ty.  independence,  quick  response  capability,  currency  in 
its  fieldCs)  of  expenise.  and  familiarity  with  the  needs  of 
its  sponsor. 

(4)  An  assessmem  of  the  adequacy  of  the  FFRDC 
management  in  ensuring  a  cost-cfTectivc  operation. 

(5)  A  determination  that  the  criteria  for  establishing 
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the  FFRDC  condnue  lo  be  satisfied  and  that  the  spon¬ 
soring  agreement  is  in  compUance  with  35.017-1. 

35ill7-5  Tenoiiiaiinf  an  FFRDC* 

When  a  sponsor's  need  for  the  FFRDC  no  longer  exists, 
the  sponsorship  may  be  transferred  to  one  or  more 
Govenmem  agencies,  if  apprepriately  justified.  If  the 
FFRDC  is  not  transferred  to  another  Government  agency,  it 
shaO  be  phased  out. 

354)174  Master  list  of  FFRDC'i. 

The  National  Science  Foundation  (NSF)  maintains  a 
master  Government  list  of  FFRDCs.  Primary  ^nsors 
will  provide  mformadon  on  each  FFRDC.  including  spon¬ 
soring  agreements,  mission  statements,  funding  data,  and 
type  of  R&D  being  performed,  to  the  NSF  upon  its  request 
for  such  information. 

354)17*7  Limitation  on  the  creation  of  new  FFRDCs. 

Pursuant  to  10  U.S.C.  2367,  the  Secretary  of  Defense, 
the  Secretary  of  the  Army,  the  Secretary  of  the  Navy,  the 
Secretary  of  the  Air  Force,  the  Secretary  of  Transportation, 
and  the  Administrator  of  the  Nation^  Aeronautics  and 
Space  Administration  may  iK>t  obligate  or  expend  amounts 
appropriated  to  the  Department  of  Defense  for  purposes  of 
operating  an  FFRDC  that  was  not  in  existence  before  June 
2.  1986,  until  (a)  the  head  of  the  agency  submits  to 
Congress  a  repon  with  respea  to  such  center  that  describes 
the  purpose,  mission,  and  general  scope  of  effort  of  the 
center,  and  (b)  a  period  of  60  days,  beginning  on  the  date 
such  report  is  received  by  Congress,  has  elapsed. 


35-8 


37 


This  page  was  left  out  of  orignial  document 


APPENDIX  D  -  OFFICE  OF  FEDERAL  PROCUREMENT  POLICY 
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CXCCUTIve  OFFICE  OF  THE  PRESIDENT 
orrtcc  or  mamaccmcnt  and  vuoocr 

w«iMM»TON,  oc  wm 


HAR  I  2  I9?t 


KCMOMNDUK  FOX /THE  ZHSFECTORS  CEHEXAt 
r*OH» 


•Mifovn  rvRyi 

■^1 


SUBJECT: 


Alltn  V.  Bunan 
Adainlttrttor 

r*d*r«Uy  Pund«4  Attorch  tnd  D«v«lopB«nt 
C«nt«rs  (FrAOCB) 


OFFF  Fplicy  lA'l,  ■Ftdtrally  Funded  Xeseareli  and  Davalopaant 

Cantara  (rrADCa)*«  as  inplaaantad  By  Part  35.017  of  tha  Faderal 
Aeouiaicien  Xefulatlon  (FAX),  previdaa  Oovarnaant'vlda  policy  for 
astabliahnant,  usa,  parlodlc  raviav  and  tarainatlon  of  tkaaa 
activttlas  Iw  Fadaral  a9anciaa.  This  aaaorandua  io  to  appriaa  you 
of  tha  FAX  eovaraga.  vhlch  vta  issuad  on  Fabruary  S«  1900,  and  to 
aak  your  help  In  aaoinp  that  thosa  pollelas  ara  obaarvad. 

Ha  particularly  want  to  aaka  aura  that  aponaorino  apanciaa  ara 
earryina  out  thair  raaponaibility  to  raviOw  tha  naad  for  thaaa 
cantara  avary  fiva  yaara.  Attached  is  a  list  of  currant  contracts 
and  a  copy  of  FAX  Subpart  9S. 017-4. 

Thank  you  for  your  aaaistanea.  Flaaaa  contact  Nr.  Xobart  Cooper  of 
ay  staff  at  303/39S-)}0e  vith  any  questions. 

Attachnant 
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W«  voul4  b«  t«  M«t  with  you  or  your,ot*f(  to 

dloeuii  thlo  aottor^  if  you  to  dooiro.  If  you  htvo  ony  question* 
rsqsrdinq  this  r*quest«  pitas*  contact  a*  at  20a-a95>Saoa  or 
Stanley  Xaufaan  of  ay  staff  at  a03-39S-(S10<  Your  cooperation 
and  assistance  aro  greatly  appreciated. 

Attachsents 

cci  Senator  Cohen 
Senator  Levin 
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EXECUTIVE  OrriCE  Of  THE  fMSIOCNT 
orriec  or  manaocmcmt  and  tvoorT 

muttmim.  o.c  am 


$CP  16  1992 


menoraxdum  rox  the  xhspectom  cemerax.  on 

DEPAMMENT  OP  DEPCHSt 
OEPAATHENT  OP  IMEKCY 

MATXOHAL  AtPOHAUTXCS  AND  SPACE  ADKIKISTRATIOK 
OEPAItTOEMT  OP  HEALTH  AMD  HUKAM  SERVICES 
MATXOHAL  SCI  DICE  POUMDATXOM 
MUCL^  RBCUUI^y  COKMISSXOM 

PROMJ  — 

AdalnlPtrAtor 

SURJECTi  P«d*rAlly  Pttnd*d  Rtstaroh  And  OAVAlopntnt  CAnttr* 

(PPRDCa) 

By  thA  AttAchAd  nAKorAndun  dAttd  MArch  13,  19*1,  X  AwrltAd 
you  oP  thA  OPflcA  of  PAdAtAl  ProcurABAnt  Policy  (OPPP)  Policy 
tottAr  SA-l,  "PPROCA*  And  Ito  InplAnAntlnc  rAgulAtory  covArAgo. 
ThA  BAAorAnduB  rAguAAtAd  your  AtolAtAnco  1b  OAolng  thAt  the 
rogulrABAntB  of  thA  Policy  LottAr  vorA  bAlng  obtArvAd. 

RAcontly,  the  SAnAtt  SubcoBBlttoA  on  Ovorolght  of  CovArnBAnt 
MAHAgABont  publlshAd  B  rAport  ontltlAd  "XnAdAquAtA  PAdArtl 
OvArslght  of  PAdArolly  PundAd  RAOAArch  And  OAVolopBAnt  ContAr**, 
ThA  rAport  dAtollAd  the  nAod  for  otrongthABAd  PodArol  controls 
ovor  PPROCs.  Raong  other  things,  the  report  recoBBAndAd  thAt 
sponsoring  sgAnclss  not  only  cospsts  contrsets  to  sstAbllsh 
PPRDCs,  but  Also  to  contlnuA  opArstion  of  sxlstlng  PPROCs.  Xt 
Also  rscoBBAndsd  thAt  OPPP  rsvisA  ths  ApproprlAts  Acqulstlon 
rsgulAtlons  to  snhAnco  cospotltlon. 

Ms  bollAVs  thAt  PPROC  contrActs  srs  Intsndsd  to  bs  long-tora 
In  nAturs.  HovAvsr,  ths  policy  rogulros  thst  AgonelAS  conduct 
AnAlysAS  prior  to  rsnsvlng  PPRDCs  to  dAtsrBlns  uhothor  ths  unigus 
PPROC  rslAtionshlp  Is  still  nsAdod.  Ths  snAlysAS  should  Addrsss 
whsthor  ths  BArkstplACA  hAS  chAngod  to  ths  point  Vhsrs 
coBpstltion  should  bs  sought.  Thsy  should  Also  svsluAts  post 
porforBAnes  And  any  future  chAngos  to  ths  PPROdi  nlsslon.  A  copy 
of  ths  SsnAts  report  is  AttAchod  for  your  InforsAtlon. 

Xn  order  to  respond  fully  to  ths  Ssnsts,  X  ab  requesting 
that  you  conduct  a  speclsl  audit  of  your  agency's  PPROC  sols 
source  renegotlAtlon  lustlfleations  devAloped  since  1919. 

Specif Ically,  the  audit  should  address  the  nature  and  adecuacy  of 
those  just IflcAt Ions,  t  wsuld  ApprsclAto  being  advised  of  ths 
results  of  your  audit  by  fsbruary  1,  19*3. 
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APPENDIX  E  -  SUMMARY  OF  PRIOR  AUDITS  AKD  OTHER  REVIEWS 
General  Account ina  Office 

Report  No.  GAO/NSIAD-88-22  (OSD  Case  No.  7551),  "Competition: 
Issues  on  Establishing  and  Using  Federally  Funded  Research  and 
Development  Centers,"  March  7,  1988.  The  report  stated  that  the 
special  relationship  FFRDCs  had  with  their  sponsors  restricted 
competition.  Due  to  the  lack  of  competition,  the  government  was 
limited  in  its  ability  to  know  whether  non-FFRDCs  could  do  work 
better  or  at  less  cost.  The  report  recommended  the  use  of  broad 
agency  announcements  on  a  test  basis  for  assessing  the  potential 
availability  of  non-FFRDCs  to  accomplish  DoD  research  and  for 
improving  DoD  assurance  that  FFRDC  work  was  the  most  effective. 
DoD  disagreed  with  the  recommendation  on  grounds  that  the  needs 
for  and  uses  of  the  FFRDCs  were  thoroughly  assessed  under 
existing  procedures. 

Inspector  General.  DoD 

Report  No.  93-113,  "DoD  Contractor  Insurance  Programs,"  June  18, 
1993.  The  report  stated  that  Defense  Logistics  Agency  contractor 
insurance/pension  reviews  resulted  in  about  $1.2  billion  of 
savings  from  FY  1990  through  the  3rd  quarter  of  FY  1992. 
However,  contractor  insurance/pension  reviews  for  an  estimated 
89  contractors  and  4  FFRDCs  that  qualified  for  reviews  were  not 
accomplished  as  required.  The  report  estimated  that  $1.6  to 
$4.4  million  in  monetary  benefits  could  be  realized  by  performing 
the  contractor  insurance/pension  reviews  at  the  Lincoln 
Laboratory,  the  Center  for  Naval  Analyses,  the  Software 
Engineering  Institute,  and  the  Logistics  Management  Institute. 
The  Director,  Defense  Logistics  Agency,  agreed  to  evaluate  the 
4  FFRDCs  to  determine  the  need  for  contractor  insurance/pension 
reviews. 

Report  No.  92-013,  "Quick  Reaction  Report  on  the  Audit  of  the 
Army  Contract  with  the  University  of  Texas  at  Austin  Institute 
for  Advanced  Technology,"  October  27,  1992.  The  report  stated 
that  the  establishment  of  the  Institute  for  Advanced  Technology 
and  its  planned  expansion  were  contrary  to  the  OFPP  policy  of 
relying  on  available  resources  for  meeting  research  needs.  The 
Army  did  not  adequately  analyze  requirements  or  properly  consider 
alternatives  to  meet  those  requirements.  The  Assistant  Secretary 
of  the  Army  (Research,  Development,  and  Acquisition)  partially 
concurred  with  the  recommendations  to  reassess  the  need  for  the 
Institute  for  Advanced  Technology  as  an  FFRDC  and  plans  to 
terminate  the  FFRDC  contract  in  FY  1994. 

Report  No.  91-115,  "Consulting  Services  Contracts  for  Operational 
Test  and  Evaluation,"  August  22,  1991.  The  report  stated  that 
the  service  contracts  used  to  support  operational  tests  were  not 
as  cost-effective  as  developing  an  in-house  capability  to  perform 
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the  work.  Services  provided  by  the  Institute  for  Defense 
Analyses  were  shown  to  cost  31  percent  more  than  the  same 
services  provided  by  comparable  in~house  personnel . 
Recommendations  were  made  to  determine  the  number  of  in-house 
personnel  needed  to  accomplish  requirements,  to  ^  make  fundinq 
adjustments  in  the  budqet  in  order  to  hire  additional  civilian 
personnel,  and  to  reduce  the  use  of  advisory  and^  assistance 
services  by  60  percent  over  the  next  5  years.  On  April  23,  1992, 
the  Deputy  Secretary  of  Defense  decided  not  to  implement  the 
recommendations  to  transfer  contract  work  to  in-house  civilian 
personnel  and  to  decrease  the  budget.  The  Deputy  Secretary  of 
Defense  requested  that  the  Assistant  Secretary  of  Defense  (Force 
Management  and  Personnel) ,  in  coordination  with  the  Secretaries 
of  the  Military  Departments,  the  Director  of  Operational  Test  and 
Evaluation,  and  the  Comptroller  of  the  Department  of  Defense, 
study  the  use  of  in-house  and  contracting  personnel  in 
operational  test  and  evaluation  activities  and  recommend  to  the 
Deputy  Secretary  of  Defense  by  July  1,  1992,  any  policy  or 

practice  changes.  In  October  1992,  the  Deputy  Secretary  of 
Defense  concluded  that  no  additional  action  would  be  taken 
pending  the  collection  and  review  by  the  Comptroller  of  the 
Department  of  Defense  of  supplementary  manpower  requirements  data 
for  the  FY  1995  budget. 

Report  No.  90-041,  "Contracting  Practices  of  the  Institute  for 
Defense  Analyses,"  March  1,  1990.  The  audit  stated  that  the 

Institute  for  Defense  Analyses  (IDA)  mission  statement  did  not 
clearly  differentiate  between  work  suitable  for  IDA  and  work  that 
a  non-FFRDC  should  perform.  Also,  program  officials  were  not 
ensuring  that  work  assigned  to  IDA  on  a  sole-source  basis  could 
not  be  done  by  non-FFRDC  sources.  The  report  recommended  that 
DDR&E  establish  a  mission  statement  for  IDA  in  accordance  with 
OFPP  Letter  84-1  and  that  DDRJeE  establish  procedures  to 
adequately  justify  the  sole-source  assignment  of  work  to  IDA. 
DDR&E  concurred  with  the  recommendations.  . 

Air  Force  Audit  Agency 

Report  No.  0056410,  "Review  of  Air  Force-Managed  Federally  Funded 
Research  and  Development  Centers,"  August  6,  1991.  The  report 

stated  that  non-FFRDC  contract  support  was  less  costly  than 
support  provided  by  the  Aerospace  and  MITRE  corporations.  Based 
on  FY  1990  data,  the  average  annual  cost  for  an  Aerospace 
Corporation  FFRDC  technical  staff  member  was  $184,000,  compared 
to  $114,000  for  a  non-FFRDC  technical  support  contract  employee. 
Likewise,  the  average  technical  staff  member  cost  at  the  MITRE 
Corporation  C^I  Division  FFRDC  was  $151,000,  compared  to  a  non- 
FFRDC  contractor  cost  of  $135,000.  The  report  stated  that  Air 
Force  Systems  Command  (now  Air  Force  Materiel  Command)  program 
managers  approved  sole— source  taskings  for  the  use  of  FFRDCs 
without  determining  whether  in-house  or  non-FFRDC  contractors 
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could  accomplish  the  vyork;  did  not  independently  develop  an 
estimate  of  their  FFRDC  support  requirements;  and  did  not 
adequately  and  objectively  evaluate  the  FFRDCs'  technical 
performance  for  any  of  the  performance  evaluations  reviewed.  Air 
Force  management  officials  agreed  with  the  report  conclusions  and 
recommendations. 

Report  No.  93064014,  Followup  Audit — Review  of  Air  Force-Managed 
Federally  Funded  Research  and  Development  Centers,"  August  18, 
1993.  The  audit  evaluated  the  effectiveness  of  management 
actions  implemented  in  response  to  selected  recommendations  in 
Report  No.  0056410,  "Review  of  Air  Force-Managed  Federally  Funded 
Research  and  Development  Centers,"  August  6,  1991.  The  report 
stated  that  Air  Force  Materiel  Command  Space  and  Missile  Systems 
Center  and  Electronic  Systems  Center  were  effectively  defining 
FFRDC  taskings,  or  using  alternative  methods,  to  prepare 
independent  estimates  of  support  requirements;  and  using  the 
independent  estimates  to  help  negotiate  fair  and  reasonable 
prices.  However,  the  Space  and  Missile  Systems  Center  had  not 
fully  implemented  procedures  to  review  and  validate  FFRDC  billing 
accuracy,  and  the  Electronic  Systems  Center  had  not  effectively 
implemented  procedures  to  evaluate  FFRDC  performance.  The  report 
contained  no  new  findings  requiring  corrective  actions. 

Department  of  the  Naw 

Assistant  Secretary  of  the  Navy  (Research,  Development,  and 
Acquisition)  Special  Procurement  Management  Review,  "Contracting 
Through  Navy  Laboratories  and  Use  of  Federally  Funded  Research 
and  Development  Centers,"  July  1989.  The  report  stated  that  Navy 
activities  were  unaware  of  the  requirements  governing  the  proper 
use  of  FFRDCs.  As  a  result,  the  Navy  inappropriately  used  Center 
for  Naval  Analyses  and  Department  of  Energy  FFRDCs  for  support 
services  and  general  information  gathering.  In  response  to  the 
inappropriate  use  of  FFRDCs,  the  Assistant  Secretary  issued  a 
May  25,  1990,  memorandum  to  remind  FFRDC  users  of  Economy  Act, 
Competition  in  Contracting  Act,  Brooks  Act,  and  FAR  35.017 
requirements . 

Congressional  Research  Service 

Report  No.  91-378  SPR,  "DoD's  Federally  Funded  Research  and 
Development  Centers  (FFRDCs),"  April  29,  1991.  The  report 
summarizes  congressional  concerns  related  to  DoD-sponsored 
FFRDCs.  These  concerns  included  the  increased  funding  and  growth 
of  the  FFRDCs  at  the  same  time  that  research,  development,  test, 
and  evaluation  funding  was  decreasing;  contentions  that 
contracting  officials  increasingly  placed  sole-source  contracts 
with  FFRDCs  to  bypass  requirements  of  the  Competition  in 
Contracting  Act  and  other  procurement  regulations;  the  extent  to 
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Which  FFRDCs  are  required  because  of  their  ability  to  maintain 
objectivity  and  avoid  conflicts  of  interest;  the  inadequate 
oversight  of  the  FFRDCs;  and  the  diversification  of  FFRDCs  into 
areas  beyond  their  originally  defined  missions.  Related  to 
diversification,  the  report  stated  that  members  of  Congress  were 
considering  allowing  some  FFRDCs  to  broaden  their  science  and 
technology  activities  together  with  requiring  them  to  compete 
with  non-FFRDCs. 

D.s.  Senate 


Subcommittee  on  Oversight  of  Government  Management,  Senate 
Committee  on  Governmental  Affuii^s  Report,  "Inadequate  Federal 
Oversight  of  Federally  Funded  Research  and  Development  Centers," 
July  8,  1992.  The  subcommittee  reported  that  cost,  accounting, 
and  auditing  controls  were  inadequate  and  inconsistent  and  had 
contributed  to  wasteful  and  inappropriate  use  of  Federal  funds  by 
FFRDCs.  The  report  stated  that  Congress  has  repeatedly  expressed 
concerns  about  DoD  excessive  funding  and  inadequate  management  of 
FFRDCs  and  noted  House  and  Senate  Appropriations  Committee 
actions  in  the  FYs  1991  and  1992  appropriations  bills  to  reduce 
funding  and  improve  the  management  of  DoD-sponsored  FFRDCs. 

The  subcommittee  report  highlighted  problems  identified  by  the 
Air  Force  Audit  Agency  and  the  Defense  Contract  ^  Audit  Agency  at 
Air  Force-sponsored  FFRDCs  involving  lack  of  independent  cost 
estimates;  failure  to  ensure  the  validity  and  reasonableness  of 
costs;  inadequate  review  of  management  fee  requests  and  travel 
costs;  inadequate  review  and  monitoring  of  projects  assigned  to 
the  FFRDCs;  and  the  charging  of  excessive  indirect  costs  by  the 
Massachusetts  Institute  of  Technology  against  the  Lincoln 
Laboratory  contract . 

The  report  addressed  subcommittee  concerns  regarding  contract 
provisions  that  allow  the  payment  of  a  management  fee  to  the 
Center  for  Naval  Analyses  for  charges  that  were  otherwise 
unallowable,  including  the  funding  of  a  tuition  program  for  the 
children  of  Center  for  Naval  Analyses  employees.  The  report 
questioned  why  a  fixed  fee  was  paid  to  a  non-profit  corporation 
whose  sole  purpose  was  to  run  an  FFRDC.  The  report  also  noted 
that  most  Defense  Contract  Audit  Agency  audits  were  not  timely 
and  several  contained  audit  exceptions  that  were  unresolved  as  of 
the  subcommittee  survey. 

The  report  stated  that  FFRDC  operating  contracts  were  generally 
not  competitively  awarded.  The  report  also  stated  that  the 
Competition  in  Contracting  Act  was  enacted  to  hold  down  contract 
costs  through  the  use  of  competitive  procedures  and  that 
exempting  FFRDCs  from  competition  under  10  U.S.C.  2304(c)(3)  was 
not  intended  to  eliminate  competition  from  contracts  to  operate 
FFRDCs . 
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The  report  addressed  concerns  about  combined  FFRDC  operations, 
stating  that  RAND  Corporation  operates  three  separate  FFRDCs  for 
DoD  and  that  MITRE  Corporation  operates  two  FFRDCs,  one  for  DoD 
and  one  for  the  Federal  Aviation  Administration.  The  report 
stated  that  both  FFRDCs  issue  combined  financial  statements  and 
provide  combined  indirect  cost  data  for  audit. 

The  report  listed  concerns  regarding  the  operation  of  "shell 
corporations,"  such  as  the  Center  for  Naval  Analyses,  the 
Institute  for  Defense  Analyses,  the  Aerospace  Corporation,  and 
the  Logistics  Management  Institute.  The  report  stated  that  the 
sole  purpose  of  the  corporations  was  to  operate  an  FFRDC  and  that 
a  shell  corporation  with  no  other  assets  or  functions  has  less 
independence  and  was  more  difficult  to  dislodge  in  the  event  of 
poor  performance. 

The  report  recommended  reforms  for  the  Office  of  Federal 
Procurement  Policy  and  the  sponsors  of  all  FFRDCs  to  improve 
oversight  over  FFRDC  spending.  The  recommendations  included 
improving  cost,  accounting,  and  auditing  controls;  re-assessing 
the  need  for  management  fees;  and  increasing  competition  for 
FFRDC  contracts.  The  report  also  recommended  that  sponsors 
assess  the  independence  of  FFRDCs  and  the  cost  and  benefits  of 
operating  multiple  FFRDCs  from  a  single  site. 
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APPENDIX  H  -  DETAILS  ON  CONTRACTS  WITH  RAND  CORPORATION  FOR 
PROJECT  AIR  FORGE.  NATIONAL  DEFENSE  RESEARCH  INSTITUTE,  AND 
ARROYO  CENTER 


Background .  The  RAND  Corporation  operates  three  DoD- 
sponsored  FFRDCs;  Project  AIR  FORCE  (formerly  Project  RAND),  the 
National  Defense  Research  Institute,  and  the  Arroyo  Center.  The 
most  recent  5-year  contracts  for  these  FFRDCs  were  awarded  on 
November  19,  1990,  for  a  total  value  of  $148.6  million;  on 
November  20,  1990,  for  a  total  value  of  $125  million;  and  on 

January  18,  1991,  for  a  total  value  of  $110  million, 
respectively.  RAND  also  conducts  non-DoD  sponsored  research 
through  its  Domestic  Research  Division. 

Project  RAND,  sponsored  by  the  Air  Force,  was  initiated  through  a 
special  contract  with  the  Douglas  Aircraft  Company  in  1946.  A 
March  9,  1946,  statement  of  work  described  the  objectives  of 

Project  RAND  as: 

A  program  of  study  and  research  on  the  broad  subject 
of  intercontinental  warfare  other  than  surface,  with 
the  objective  of  recommending  to  the  Army  Air  Forces 
preferred  technic[ue  and  instrumentalities  for  this 
purpose. 

The  concept  of  Project  RAND  was  based  on  the  desire  to  retain  the 
services  of  civilian  scientists  for  Government  and  military 
activities  after  World  War  II  to  assist  in  military  planning  and 
especially  to  coordinate  such  planning  with  research  and 
development  decisions.  The  broad  nature  and  inf luence  of  the 
mission  of  Project  RAND  led  to  concerns  about  conflicts  of 
interest.  This  concern  led  to  incorporation  of  Project  RAND  as  a 
non-profit  corporation  in  May  1948,  and  in  November  1948,  the 
Project  RAND  contract  was  transitioned  to  the  RAND  Corporation. 
In  1976,  Project  RAND  was  renamed  Project  AIR  FORCE.  All  Air 
Force  contracts  for  Project  RAND/Project  AIR  FORCE  were  awarded 
sole-source  to  the  RAND  Corporation. 

The  National  Defense  Research  Institute  (NDRI)  was  established  at 
the  request  of  the  RAND  Corporation  in  1984.  NDRI  was 

established  to  consolidate  on-going  research  and  development  work 
conducted  by  RAND^s  Defense  Manpower  Research  center  and  Strategy 
Assessment  Center  for  various  OSD  elements. 

The  Arroyo  Center,  an  Army-sponsored  FFRDC,  was  transferred  in 
1984  to  the  RAND  Corporation  at  the  request  of  the  Secretary  of 
the  Army  and  Chief  of  Staff  of  the  Army.  The  Arroyo  Center  was 
originally  established  in  1982  at  the  Jet  Propulsion  Laboratory, 
an  FFRDC  sponsored  by  the  National  Aeronautics  and  Space 
Administration  and  operated  by  the  California  Institute  of 
Technology.  The  1984  contract  and  the  1989  contract  were 
sole-source  awards  to  the  RAND  Corporation. 
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In  addition  to  DoD-sponsored  work  conducted  through  FFRDCs,  RAND 
also  conducts  research  and  studies  for  more  than  70  non-DoD 
sponsors,  including  other  Federal  Government  agencies,  state  and 
local  governments,  private  corporations,  universities,  and 
foundations.  The  non-DoD  research  amounts  to  20  to  25  percent  of 
RAND  annual  workload. 

Basis  for  Project  AIR  FORCE  contract  renewals  The 
1990  contract  was  awarded  sole-source  to  RAND  Corporation  by  the 
Air  Force  Office  of  Scientific  Research.  A  comprehensive  review 
document  addressing  the  OFPP  and  FAR  criteria  on  continued  need 
was  not  prepared  for  Project  AIR  FORCE.  The  noncompetitive 
5-year  contract  was  based  on  an  acquisition  plan  ^  and  ^  a 

sole-source  justification  prepared  by  the  Office  of  Scientific 
Research.  The  10-page  acquisition  plan  addressed  the  acquisition 
history  and  objective  of  Project  AIR  FORCE,  the  basis  for 
selection,  the  contracting  considerations,  and  the  milestones  for 
the  procurement  cycle.  The  plan  stated: 

The  initial  reasons  for  establishing  the  RAND 

Corporation  are  still  valid.  The  problems  of  national 
security  continue  to  grow  in  complexity,  and  the  years 
since  the  founding  of  RAND  have  been  witness  to  a 
technological  explosion. .. .RAND  is  recognized  as 
having  made  unique  contributions  to  national 
security... 


The  services  to  be  provided  by  Project  AIR  FORCE  (PAP) 
consist  of  a  program  of  study  and  research  on  the 
broad  subject  of  aerospace  power  with  the  object  of 
recommending  to  the  United  States  Air  Force  preferred 
methods,  techniques,  and  instrumentality  for  the 
development  and  employment  of  aerospace  power.  The 
Air  Force  requirement  for  the  type  of  policy  and 
systems  analysis  support  currently  provided  by  PAP  is 
expected  to  continue  for  an  indefinite  period  of  time. 
Executive  level  officials  have  continually  expressed 
support  for  this  type  of  service.  The  services  to  be 
provided  by  PAF  can  be  described  as  timely  and 
objective  research  on  methods,  techniques,  and 
instrumentalities  for  the  development  and  employment 
of  aerospace  power  for  the  Air  Force. 


The  delivery  of  required  reports,  analyses,  etc.  will 
be  governed  by  individual  tasking  arrangements. 
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The  acquisition  plan  stated  the  authority  for  the  sole-source 
contract  was  FAR  6.302-3  and  that  the  acquisition  would  not  be 
synopsized  in  the  Commerce  Business  Daily  because 
FAR  5.202 (a) (10)  permitted  the  contracting  officer  to  waive  the 
requirement.  The  plan  advocated  a  5-year,  cost-plus  fixed-fee 
contract  because: 

...the  mix  of  various  labor  categories  required  for 
each  individual  program  cannot  be  forecast  since  only 
general  areas  to  be  studied  are  known  in  advance. 

Specific  topics  to  be  studied  under  each  program  are 
determined  as  the  requirements  evolve  during  the 
period  of  performance.  Under  these  circumstances,  it 
is  not  possible  to  reasonably  predict  these 
requirements  at  the  time  of  negotiations  so  that 
realistic  individual  project  levels-of-ef fort  can  be 
estimated. 

In  regard  to  management  control,  the  acquisition  plan  stated: 

The  Air  Force  Advisory  Group  (AFAG)  for  Project  AIR 
FORCE  is  composed  of  fourteen  Air  Staff  general 
officers  and  senior  civilian  officials. .. .The  AFAG 
conducts  substantive  reviews  of  selected  research 
projects  during  periodic  meetings  with  RAND 
management ... .Each  project  initiated  under  Project  AIR 
FORCE  is  sponsored  by  a  HQ  USAF  or  MAJCOM  [Major 
Command]  general  or  Senior  Executive  Service  (SES) 
of f icer . . . . The  sponsor  provides  guidance  on  Air  Force 
needs  and  priorities,  periodically  reviews  the  status 
of  the  research  effort  and  assumes  the  lead  for 
implementing  those  findings  and  recommendations 
accepted  by  the  Air  Force. 

The  two-page  sole-source  justification  document  described  the 
services  to  be  procured  as  follows. 

The  requirement  is  for  150  work  years  of  professional 
effort  each  year.  The  statement  of  work  for  services 
is  to  perform  a  program  of  study  and  research  on  the 
broad  subject  of  Aerospace  power  with  the  objective  of 
recommending  to  the  United  States  Air  Force  preferred 
methods,  techniques,  and  instrumentalities  for  the 
development  and  employment  of  Aerospace  Power. 


55 


APPENDIX  H  -  DETAILS  ON  CONTRACTS  WITH  RAND  CORPORATION  FOR 
PROJECT  AIR  FORCE.  NATIONAL  DEFENSE  RESEARCH  INSTITUTE >  AND 
ARROYO  CENTER  (cont'd) 

The  July  5,  1990,  justification  document  cited  FAR  6.302-3  as  the 
authority  for  the  sole-source  procurement  and  stated  this 
authority  was  applied  because: 

Contracting  with  The  RAND  Corporation  is  consistent 
with  Air  Force  Policy. ..and  OSD's  decision  to  retain 
Project  AIR  FORCE  as  a  FFRDC.  The  RAND  Corporation  is 
recognized  as  having  made  unigue  contributions  to 
National  Security,  assisting  the  Air  Force  in  its  role 
of  leadership  in  applying  modern  analytic  techniques. 

The  justification  also  stated  that,  based  on  FAR  5.202 (a) (10) ,  no 
synopsis  would  be  published  in  the  Commerce  Business  Daily  and  no 
further  steps  would  be  taken  to  obtain  competition.  The 
justification  further  stated  that  an  informal  market  survey  had 
determined  that  no  other  acceptable  qualified  sources  could 
satisfy  the  requirements  and  that  no  other  ^  contracting  sources 
had  expressed  written  interest  in  this  acquisition. 

Basis  for  NDRI  contract  renewal.  On  November  30,  1989 

(effective  October  1,  1989),  DSS-W  noncompetitively  awarded  the 

contract  for  NDRI  to  the  RAND  Corporation.  A  periodic  review 
document  and  sole-source  justification  were  the  basis  for  the 
award  to  RAND.  The  Director,  DoD  FFRDC  Programs,  DDR&E,  prepared 
both  documents.  On  August  14,  1989,  DDR&E  approved  the  sole- 
source  justification.  On  November  8,  1989,  the  Deputy  DDR&E 

(Research  and  Advanced  Technology)  approved  the  review  document. 
The  10-page  periodic  review  document  described  the  ^  background  of 
NDRI,  the  RAND  environment,  research  themes  and  directions,  and 
FFRDC  organization.  The  review  document  stated  that  NDRI's 
research  agenda  was  generally  oriented  towards  resolving 
three  key  defense  policy  questions. 

o  How  is  the  global  evolution  of  military  power, 
political  influence,  and  economic  strength  affecting  the 
challenges  faced  by  the  United  States? 

o  How  can  force  employment  strategy,  weapon  system 
acquisition,  logistics  support,  and  personnel  force  evolution  be 
better  planned  and  managed? 

o  How  should  U.S.  forces  be  modernized  to  assure 
continual  deterrence,  particularly  with  respect  to  conventional 
balance  of  forces  in  Europe? 

The  Director,  DoD  FFRDC  Programs,  requested  16  current  senior- 
level  DoD  officials  who  sponsored  studies  and  10  former 
DoD  senior-level  officials  who  had  sponsored  studies  when  they 
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were  in  Government  service  for  assessments  of  the  use  and  need 
for  NDRI.  The  May  12,  1989,  memorandum  to  the  current  officials 
stated : 

As  reference,  copies  are  attached  of  NDHI's  1985, 

1986,  and  1987  annual  reports  (the  1988  edition  will 
be  mailed  to  you  soon),  as  well  as  a  list  of  NDRI's 
projects  during  the  last  five  years.  Instead  of  a 
project-by-project  evaluation,  it  would  be  helpful  if 
your  input  is  more  of  an  overall  assessment  of  the 
continuing  need  for  the  FFRDC,  the  effectiveness  of 
NDRI  in  meeting  its  sponsors'  needs  for  objective 
research  and  analysis,  and  the  adequacy  of  the  FFRDC 
management . 

The  May  12,  1989,  letter  to  former  officials  stated: 

Your  judgment  is  solicited  on  the  continuing  need  for 
this  FFRDC  as  well  as  on  the  extent  to  which  the  NDRI 
has  met  its  objectives.  References  to  notable 
capabilities  and/or  products  would  be  appreciated.  A 
copy  of  the  latest  NDRI  Annual  Report  is  enclosed; 
earlier  ones  are  available  on  request. 

All  26  responders  supported  renewal  of  the  NDRI  contract. 
However,  all  of  the  current  officials  were  nonspecific  regarding 
their  future  requirements.  Regarding  alternative  sources,  2  of 
the  16  sponsors  stated  that  the  FFRDCs  performed  only  appropriate 
tasks  since  each  project  was  reviewed  to  determine  whether  the 
project  should  be  performed  by  the  FFRDC  or  some  other 
organization.  The  other  sponsors  either  did  not  address  this 
area  or  discussed  the  close  relationship  between  the  FFRDC  and 
their  organization.  The  responses  from  current  senior- level 
OSD  officials  were  mixed  on  the  effectiveness  and  efficiency  of 
the  FFRDC  in  meeting  sponsor  needs.  Several  commented  negatively 
on  NDRI's  lack  of  responsiveness,  particularly  on  delivery  of 
reports.  One  official  stated  that,  "NDRI's  work  is  not 
appreciably  better  as  far  as  quality  of  research  than  many  non- 
FFRDCs."  Other  officials  believed  that  the  FFRDC  was  effective 
and  responsive  to  performing  their  work.  Most  of  the  former  and 
current  officials  stated  that  NDRI  management  was  supportive  of 
the  sponsor's  requirements.  The  officials  generally  believed 
that  the  NDRI  adhered  to  the  criteria  for  an  FFRDC  and  the 
sponsoring  agreement,  as  required  by  FAR  35.017-1. 

The  two-page  sole-source  justification  for  the  NDRI  contract 
described  the  services  to  be  provided  as  follows. 
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...the  broad  objectives  of  the  work  to  be  performed 
under  this  contract  are  to  (1)  provide  expert  and 
independent  interdisciplinary  research  capabilities 
covering  a  broad  range  of  relevant  specialties;  (2) 
enhance  mechanisms  for  the  technology  transfer  among 
Office  of  the  Secretary  of  Defense  components;  and  (3) 
further  institutionalize  capabilities  for  analysis  and 
integration  of  defense  issues  that  cut  across  the 
responsibilities  of  individual  DoD  components.  A  wide 
range  of  research,  studies  and  analyses  are  expected 
to  be  conducted  in  eight  areas:  (1)  applied  science 
and  technology;  (2)  defense  manpower  research;  (3) 
information  processing  systems;  (4)  international 
economic  policy  as  it  relates  to  defense  policy;  (5) 
international  security  and  defense  policy;  (6) 
acquisition  and  support  policies;  (7)  force 
employment;  and  (8)  strategy  planning  and  assessment. 

The  sole-source  justification  cited  FAR  6.302-3  as  the  authority 
for  the  sole-source  procurement,  stating  that  NDRI  had  developed 
a  unique  capability  to  support  the  OSD,  the  Joint  Staff,  and  the 
Defense  Agencies  in  areas  that  could  not  be  performed  by  a  non- 
FFRDC.  The  justification  stated  that  a  market  survey  was  not 
conducted  or  necessary  because  NDRI  was,  ’'a  unique  organization 
that  cannot  be  duplicated."  It  further  stated,  "A  listing  of  the 
sources,  if  any,  that  expressed,  in  writing,  an  interest  in  the 
organization  is  not  applicable  to  FFRDC."  The  sole-source 
justification  was  approved  by  the  Director  of  Contracting,  DSS-W, 
and  the  DDR&E. 

Synopsis  of  NDRI  contract  in  Commerce  Business  Daily*  On 
June  29,  1989,  DSS-W  published  a  notice  of  the  proposed 
sole-source  award  to  NDRI  in  the  Commerce  Business  Daily.  The 
synopsis  stated: 

CONTimilNG  RESEARCH  IN  SUPPORT  OP  OSD  AND  0JCS....The 
Dept,  of  Defense  plans  to  award  a  sole-source  contract 
to  the  Rand  Corporation. .  .for  work  to  be  performed  by 
the  National  Defense  Research  Institute  (NDRI),  a 
Federally  Funded  Research  and  Development  Center 
(FFRDC) .  This  contract  is  for  continuation  of 
multiple-year  research  currently  sponsored  by  various 
elements  of  the  Office  of  the  Secretary  of  Defense 
(OSD),  office  of  the  Joint  Chiefs  of  Staff  (OJCS),  The 
Defense  Advanced  Research  Projects  Agency,  and  other 
Defense  Agencies.  The  broad  objectives  of  the  NDRI 
program  are  to  provide  expert  and  independent  inter¬ 
disciplinary  research  capabilities  covering  a  broad 
range  of  relevant  specialties  and  further  develop  and 
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institutionalize  capabilities  for  analysis  and 
integration  of  defense  issues  that  cut  across  the 
responsibilities  of  individual  DoD  components.  A  wide 
range  of  research  is  expected  to  be  conducted  in  the 
following  areas:  defense  research  management, 

technology  and  information  processing,  international 
security  and  policy,  strategy  and  force  employment. 

The  contracting  office  received  no  inquiries  regarding  this 
synopsis. 

Basis  for  Arrovo  Center  contract  renewal.  DSS-W  awarded  the 
January  8,  1991,  sole-source  contract  for  the  Arroyo  Center.  The 
effective  date  of  the  contract  was  October  1,  1990.  The  Army 

could  not  provide  a  comprehensive  review  that  addressed  OFPP 

Letter  84-1  and  FAR  35.017-4  criteria.  To  support  the 
noncompetitive  procurement,  the  Director  of  Management,  Office  of 
the  Chief  of  Staff  of  the  Army,  polled  Army  Major  Commands  and 
the  Army  Staff  on  their  use  of,  and  need  for,  the  Arroyo  Center, 
and  prepared  a  sole-source  justification  dated  August  9,  1990. 

Users  responding  to  the  request  for  comments  included  the 

Commanding  General,  Forces  Command;  the  Commanding  General, 
Training  and  Doctrine  Command;  the  Commanding  General,  Army 
Materiel  Command;  the  Deputy  Chief  of  Staff  for  Operations  and 

Plans,  Department  of  the  Army;  and  three  Directors  within  the 
Office  of  the  Deputy  Chief  of  Staff  for  Operations  and  Plans, 
Department  of  the  Army.  The  three  Commanding  Generals  and  the 
Deputy  Chief  of  Staff  for  Operations  and  Plans  were  members  of 
the  Arroyo  Center  Policy  Committee,  which  met  periodically  to 
review  and  approve  the  Arroyo  Center  program. 

All  inputs  from  the  users  and  members  of  the  Arroyo  Center  Policy 
Committee  rated  Arroyo  Center  services  favorably  and  supported 
renewal  of  the  contract.  The  memorandum  from  the  Commanding 
General,  Forces  Command,  stated: 

While  I  cannot  predict  definitive  Army  rec[uirements 
and  internal  assets  in  the  future,  it  stands  to  reason 
that  the  Army  will  have  a  continuing  need  for  thorough 
and  sound  analysis  on  topics  which  will  require 
technical  expertise  and/or  objective  perspectives. 

There  are,  of  course,  a  number  of  alternative  sources 
for  this  kind  of  research;  however,  the  Arroyo 
Center's  links  to  the  RAND  Corporation  and  its  track 
record  of  outstanding  service  to  the  Army  should  argue 
in  favor  of  renewing  the  Center's  contract. 
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The  memorandum  from  the  Commanding  General,  Training  and  Doctrine 
Command ,  stated : 

Assessments  of... Arroyo  Center's  work... vary  between 
individual  sponsors. ...  from  unqualified  "thumbs-up"  to 
qualified  "thumbs-down". .. .TRADOC  has  a  continuing 
requirement  for  analytical  support,  particularly  in 
the  functional  areas  of  training  and  combat 
developments. .. .TRADOC  does  not  depend  solely  on  the 
Arroyo  center  for  research-related  support.  Organic 
assets . , . and  various  contractors  are  employed  to 
support  TRADOC ' s  research  needs .... TRADOC  has  a 
continuing  need  for  Arroyo  Center  support,  and 
therefore  I  recommend  the  Army's  contract  with  the 
Arroyo  Center  be  renewed. 

The  memorandum  from  the  Commanding  General,  Army  Materiel 
Command ,  stated : 

The  RAND  support  in  the  development  of  logistics 
concepts  and  supporting  management  systems  and 
ammunition  programming  and  budgeting  has  been 
exceptional. .. .However,  they  have  been  less  successful 
in  providing  assistance  in  the  Army  technology  base. 


Because  of  the  declining  resources  and  changing  force 
levels  in  the  Army  and  AMC,  the  need  for  innovative 
thinking  and  creative  solutions  that  are  available 
through  Arroyo  Center  is  greater  today  than  when  they 
were  formed. 

The  memorandum  from  the  Deputy  Chief  of  Staff  for  Operations  and 
Plans  stated: 

The  Army  needs  the  Arroyo  Center.  Arroyo  Center 
provides  senior  Army  leadership  with  research  and 
analysis  that  has  proven  to  be  useful,  relevant  and 
credible.  Arroyo  Center's  objective  and  independent 
research  on  national  defense  policy  issues  is  unique, 
cannot  be  obtained  in-house,  and  is  best  provided  by  a 
Federally  Funded  Research  and  Development  Center 
(FFRDC)  operation. 
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With  regard  to  the  Army's  special  technical  needs  and  mission 
requirements,  the  Deputy  Assistant  Deputy  Chief  of  Staff  for 
Operations  and  Plans,  Force  Development,  made  the  following 
comments : 


RAND  Arroyo  maintains  a  full  range  of  capabilities 
that  can  support  Army  requirements  e.g.,  theater  level 
warfight  analysis  to  system  specific  COBA's  (cost  and 
operational  effectiveness  analysis].  They  have 

provided  such  support  when  organic  Army  analysis 
agencies  (CAA  and  TRAC)  (Concepts  Analysis  Agency  and 
TRADOC  Analysis  Command)  are  fully  committed. 

In  addressing  the  issue  of  alternative  sources  to  meet  the  Army's 
needs,  the  Deputy  Assistant  Deputy  Chief  of  Staff  for  Operations 
and  Plans,  Force  Development,  further  stated: 

Using  internal  Army  analysis  organizations  is  normally 
attempted,  however,  the  required  workload  usually  far 
exceeds  the  ability  of  current  internal  agencies  to 
provide  the  necessary  support.  RAND  in  many  cases  has 
been  very  responsive  and  adaptive  to  special 
unprogrammed  study  projects. 

The  sole-source  justification  document  provided  the  following 
description  for  the  services  to  be  procured  from  the  Arroyo 
Center : 


, . * ( 1 )  provide  expert  and  independent 

interdisciplinary  analytical  research  capabilities 
covering  a  broad  range  of  relevant  specialties;  (2 ) 
further  institutionalize  capabilities  for  analyses  and 
integration  of  Army  issues  that  cut  across  the 
responsibilities  of  individual  Army  components;  and 
(3)  enhance  mechanisms  for  the  transfer  of  information 
across  service  components.  A  wide  range  of  research, 
studies  and  analyses  are  expected  to  be  conducted  in 
the  following  areas:  (1)  strategy  planning  and 
assessment;  (2)  force  design  and  structure;  (3)  force 
operations;  (4)  readiness  and  support  infrastructure; 

(5)  acquisition  and  support  policies;  (6)  applied 
science  and  technology  applications;  (7) 
methodological  development;  (8)  manpower,  training, 
and  performance;  (9)  threat  assessment;  and  (10)  Army 
policies  and  doctrine. 

The  authority  cited  for  the  sole-source  procurement  was 
FAR  6.302-3.  The  justification  stated  "...a  market  survey  was 
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not  necessary  since  the  Arroyo  Center  is  a  unique  organization 
that  cannot  be  duplicated."  The  justification  further  stated 
"...replacement  of  the  unique  capability  provided  by  the  Arroyo 
Center  would  be  very  costly  to  the  Government  in  terms  of  dollars 
and  delay  in  meeting  DA  [Department  of  the  Army]  requirements." 

Synopsis  of  Arrovo  Center  contract  jp  Business 

Daily.  DSS-W  published  a  notice  of  proposed  sole-source  award  to 
the  Arroyo  Center  in  the  Commerce  Business  Daily  on  July  6,  1990. 
The  synopsis  stated: 

CONTINUING  RESEARCH  IN  SUPPORT  OF  SHE  U.S. 

ARMY _ DSS-W  intends  to  award  a  sole  source  contract 

to  Rand  Corporation. .. for  the  continuation  of 
multiple-year  research,  studies,  and  analyses 
currently  sponsored  by  various  elements  of  the  Dept, 
of  the  Army.  See  Note  22.  (Note  22  stated:]  The 
proposed  contract  action  is  for  supplies  or  services 
for  which  the  Government  intends  to  solicit  and 
negotiate  with  only  one  source  under  authority  of 
FAR  6.302.  Interested  parties  may  identify  their 
interest  and  capability  to  respond  to  the  requirement 
or  submit  proposals.  This  notice  of  intent  is  not  a 
request  for  competitive  proposals. . .  .A  determination 
by  the  Government  not  to  compete  this  proposed 
contract  based  upon  responses  to  this  notice  is  solely 
within  the  discretion  of  the  government.  Information 
received  will  normally  be  considered  solely  for  the 
purpose  of  determining  whether  to  conduct  a 
competitive  procurement. 

We  did  not  identify  any  inquiries  from  other  organizations  in 
response  to  the  synopsis. 

Evaluation  of  the  reviews,  sole-source  juatif ications.  and 
synopses .  We  concluded  that  the  renewal  of  contracts  for  the 
three  FFRDCs  operated  by  RAND  were  not  adequately  justified. 
Comprehensive  review  documents  meeting  the  OFPP  and  FAR  criteria 
on  continuing  need  for  FFRDCs  were  not  prepared  for  Project 
AIR  FORCE  and  the  Arroyo  Center.  The  review  for  NDRI  was 
prepared  and  approved  after  the  sole-source  justification  and  the 
effective  date  (October  1,  1989)  of  the  contract.  The 

requirements  for  Project  AIR  FORCE,  NDRI,  and  Arroyo  Center  were 
stated  in  broad,  general  terms.  Based  on  the  descriptions  in  the 
review  documents,  acquisition  plans,  and  sole-source 
justifications  of  the  work  to  be  performed,  each  FFRDC  could  be 
tasked  to  perform  a  study  on  almost  any  subject  of  interest  to 
the  FFRDC 's  sponsor.  None  of  the  review  documents,  acquisition 
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plans,  or  justifications  articulated  the  missions  of  Project  AIR 
FORCE,  NDRI,  and  the  Arroyo  Center  clearly  enough  to 
differentiate  their  work  from  studies  performed  under  contracts 
for  advisory  and  assistances  services,  or  by  DoD  personnel,  or  by 
another  FFRDC.  Market  surveys  were  not  formally  conducted  to 
identify  potential  alternatives  to  Project  AIR  FORCE,  NDRI,  or 
the  Arroyo  Center.  The  proposed  sole-source  awards  were 
synopsized  in  the  Commerce  Business  Daily  for  NDRI  and  the  Arroyo 
Center . 

The  Air  Force  and  the  Army  did  not  address  the  efficiency  and 
effectiveness  of  Project  AIR  FORCE  and  the  Arroyo  Center  in  the 
acquisition  plans  or  the  sole-source  justifications.  However, 
annual  reviews  of  the  performance  of  Project  AIR  FORCE  and  the 
Arroyo  Center  were  performed.  Users  of  Project  AIR  FORCE 
completed  a  1-page  questionnaire  that  asked  12  questions  related 
to  the  quality,  utility,  value,  and  timeliness  of  work  done  by 
Project  AIR  FORCE.  Each  question  provided  for  a  check-mark 
rating  of  "Exceeds,"  "Meets,"  or  "Fully  Short  of  Expectations." 
The  Air  Force  Director  of  Plans  summarized  the  FY  1990  review 
results  in  a  October  22,  1990,  letter  to  the  Vice  President, 
Project  AIR  FORCE.  The  Director  of  Plans  stated  "...RAND  met  or 
exceeded  expectations  on  98  percent  of  the  assessments...." 
Users  of  approved  Arroyo  Center  projects  were  asked  to  complete  a 
three-page  evaluation  guideline  to  assess  the  work  done  by  the 
Arroyo  Center  before  award  of  the  January  19,  1991,  contract. 
The  users  were  asked  to  provide  multiple-choice  responses  to 
30  statements  related  to  the  usefulness,^  credibility,  and 
presentation  of  Arroyo  Center  work.  The  possible  responses  were, 
"Unable  to  evaluate,"  "Strongly  Disagree...,"  "Disagree...," 
"Agree...,"  and  "Strongly  Agree  with  statement."  The  Army  was 
not  able  to  provide  the  final  user  results  of  the  survey. 
However,  in  a  May  16,  1990,  memorandum  to  the  Assistant  Secretary 
of  the  Army  (Research,  Development,  and  Acquisition) ,  the  Army 
Director  of  Management  stated  that  "...responses  [to  the 
survey] .. .were  favorable."  The  survey  of  current  and  past 
senior-level  users  of  NDRI  covered  multiple  projects  over  several 
years.  The  reviews  of  the  use  of  Project  AIR  FORCE,  the  Arroyo 
Center,  and  NDRI  were  not  adequate  assessments  of  the  efficiency 
and  effectiveness  of  the  FFRDCs  because  user  ratings  were  based 
on  subjective  criteria. 

The  review  documents  also  did  not  discuss  what  controls  had  been 
established  by  RAND  to  ensure  a  cost-effective  operation.  The 
review  documents,  acquisition  plans,  and  justifications  also  did 
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not  establish,  for  any  of  the  three  FFRDCs,  that  the  FAR  criteria 
for  establishing  FFRDCs  continued  to  be  satisfied  and  that  the 
sponsoring  agreements  complied  with  the  FAR. 

Several  DoD  activities  provided  comments  that  were  not  addressed 
in  the  review  documents  and  justifications,  but  appear  to  be 
relevant.  For  example,  a  memorandum  prepared  by  an  official  in 
the  Office  of  the  Under  Secretary  of  Defense  for  Policy,  one  of 
the  primary  users  of  NDRI,  raises  questions  about  the  basis  for 
establishing  the  NDRI  and  its  efficiency  and  effectiveness  and 
cost  controls.  The  memorandum  stated  that: 

...the  initiative  for  establishing  the  NDRI  (in  1984] 
came  from  RAND,  not  the  Government,  and  the  logic  for 
doing  so  was  not  administrative. ..  .RAND's  main 
argument  for  the  facility  was  that  stabilized  FFRDC 
funding  is  essential  if  one  intends  RAND  to  maintain 
staff  expertise  and  data  bases  in  certain  specialized 
fields  for  which  the  DoD's  requirements  are  occasional 
or  intermittent.  Additionally,  RAND  argued  that  the 
NDRI  would  facilitate  a  more  coherent  research  agenda, 
conscious  of  its  various  aspects  and  lending  itself  to 
integration  of  or  cross-fertilization  between 
similarly  focused  projects  for  different  sponsors. 

OSD  accepted  those  arguments,  but. . .neither  [argument] 
is  compelling  based  on  actual  experience. 


At  the  same  time,  Rand  research  under  the  NDRI 
arrangement  is  more  costly  than  was  the  case  prior  to 
establishment  of  the  FFRDC,  whether  measured  in  cost 
rates  or  expressed  as  the  percentage  of  each  contract 
dollar  actually  made  available  to  approved  projects. 
Additional  cost  factors  derive  from  set-aside  of  funds 
for  "project  development,”  management  layering  (i.e., 
the  creation  of  an  NDRI  oversight  and  administrative 
bureaucracy  that  plays  no  role  in  substantive 
research),  and  correspondingly  higher  overhead  costs. 


...the  NDRI  FFRDC  and  its  dedicated  funding 
arrangement  have  produced  a  costly  and  essentially 
anti-competitive  procurement  convenience,  which  has 
largely  failed  to  arrest  or  repair  the  decline  in  the 
quality  of  Rand's  research  in  certain  high  visibility 
fields  of  interest. . .and  has  produced  little  of  value 
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in  the  area  of  research  integration  through  so-called 
"cross-functional  studies."  The  fact  that  performance 
has  not  deteriorated  in  some  other  fields  is 
insufficient  reason  to  fund  the  NDRI. 

The  comments  received  from  several  users  of  the  Arroyo  Center 
indicated  that  the  Arroyo  Center  performed  studies  and  analyses 
that  the  Army  in-house  organizations  were  unable  to  perform 
because  of  staffing  limitations.  The  person  ^  preparing  the 
sole-source  justification  for  the  Arroyo  Center  did  not  docximent 
why,  if  additional  staffing  was  authorized,  the  work  performed  by 
the  Arroyo  Center  could  not  be  performed  by  in-house 
organizations  such  as  the  Army  Concepts  Analysis  Agency  or  the 
Army  Training  and  Doctrine  Command's  Analysis  Center.  The 
justification  also  did  not  document  why  the  work  could  not  be 
contracted  to  private-sector  firms  or  other  FFRDCs.  We  concluded 
that  the  justifications  for  the  three  FFRDCs  at  the  RAND 
Corporation  were  not  adequate  for  the  award  of  the  three  5-year 
sole-source  contracts  with  a  total  estimated  cost  of 
$383.6  million. 

Management  comments  and  Audit  Response 

Management  comments .  The  Deputy  DDR&E  recommended  deletion 
of  the  statement  “The  reviews  of  the  use  of . . .  NDRI  were  not 
adequate  assessments  of  the  efficiency  and  effectiveness  of  the 
FFRDC  because  user  ratings  were  based  on  subjective  criteria.” 
The  Deputy  DDR&E  stated  that  many  studies  performed  by  the  FFRDCs 
are  inherently  subjective  and  designing  quantitative  criteria 
would  often  simply  quantify  subjective  arguments  without  adding 
to  their  validity. 

Audit  response.  We  did  not  delete  the  statement.  The  DoD 

Management  Plan  for  FFRDCs  requires  the  primary  sponsor  for 

each  FFRDC  to  review  and  report  on  the  costs  and  value  to 
DoD  of  goods  and  services  provided  by  the  FFRDCs. 
Sponsoring  agencies  should  have  sufficient  in-house 
expertise  to  objectively  evaluate  the  work  performed  by  the 
FFRDCs,  and  performance  measures  to  assess  inputs,  outputs, 
efficiency,  and  effectiveness  for  the  FFRDCs.  The 

performance  measures  must  be  verifiable  and  results  oriented 
and  should  include  qualitative  as  well  as  quantitative 
measures.  Subjective  assessments  can  present  distorted 
pictures.  Also,  the  answer  keys  used  on  the  assessments 
were  different.  If  DoD  is  going  to  use  subjective 

questionnaires  as  a  portion  of  the  reviews  of  FFRDCs,  the 

DoD  should  at  least  standardize  the  answer  keys  used. 
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APPENDIX  I  -  DETAILS  ON  CONTRACT  WITH  INSTITUTE  FOR  DEFENSE 
ANALYSES 

Background .  The  Institute  for  Defense  Analysis  (IDA)  was 
incorporated  as  a  non-profit  organization  in  April  1956.  IDA^s 
establishment  resulted  from  a  1955  request  by  the  Secretary  of 
Defense  and  the  Joint  Chiefs  of  Staff  for  an  organized  university 
effort  as  a  public  service  to  support  and  strengthen  the  Weapons 
Systems  Evaluation  Group.  The  Weapons  Systems  Evaluation  Group 
was  established  in  1949  as  a  staff  division  to  the  Joint  Chiefs 
of  Staff  to  provide  information  on  the  operational  significance 
of  new  weaponry.  The  second  Hoover  Commission  reviewing  the 
operations  of  the  Organization  of  the  Joint  Chiefs  of  Staff 
approved  the  concept  of  the  Weapons  Systems  Evaluation  Group  but 
noted  that  the  staff  was  too  small  and  that  efforts  to  expand  the 
staff  had  been  unsuccessful  because  of  restrictions  on  civil 
service  pay  and  on  the  size  of  the  Joint  Chiefs  of  Staff.  The 
Hoover  Commission  recommended  that  a  more  adequate  staff  be 
developed  through  a  private  organization.  The^  IDA  Board  of 
Trustees  was  originally  made  up  of  administrators  from 
five  educational  institutions:  California  Institute  ^  of 
Technology,  Case  Institute  of  Technology,  Massachusetts  Institute 
of  Technology  (MIT) ,  Stanford  University,  and  Tulane  University 
of  Louisiana.  It  was  anticipated  that  the  university  association 
as  established  would  lead  to  close  relations  between  IDA  and  the 
university  faculties. 

Since  IDA'S  inception  in  1956,  contracts  awarded  to  IDA  have 
never  been  competed.  IDA  currently  provides  studies  and 
analyses,  systems  engineering,  and  laboratory  and  test  support  to 
more  than  25  DoD  organizations,  including  the  Office  of  the 
Secretary  of  Defense,  the  Joint  Chiefs  of  Staff,  the  European 
Command,  the  Transportation  Command,  the  Special  Operations 
Command,  the  Advanced  Research  Projects  Agency,  the  Defense 
Information  Systems  Agency,  the  Defense  Nuclear  Agency,  the 
Ballistic  Missile  Defense  Organization,  and  the  National  Security 
Agency.  IDA  also  provides  support  to  such  non-Defense  activities 
as  the  Federal  Bureau  of  Investigation,  the  Arms  Control  and 
Disarmament  Agency,  and  the  Congressional  Budget  Office. 

Basis  for  1988  sole-source  contract.  The  Director, 
DoD  FFRDC  Programs,  DDR&E,  prepared  a  review  document  dated 
December  26,  1989.  The  sole-source  justification  to  support  the 
renewal  of  the  FFRDC  contract  with  IDA  was  dated  June  9,  1988. 
DSS-W  awarded  the  contract  on  December  22,  1988,  with  an 
effective  date  of  October  1,  1988. 

A  July  25,  1988,  DDR&E  memorandum  requested  officials  in 
24  offices  within  the  OSD,  the  Organization  of  the  Joint  Chiefs 
of  Staff,  and  the  Defense  Agencies  to  provide  comments  to  support 
renewal  of  the  contract.  Each  of  the  offices  had  tasked  IDA  to 
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perform  studies  or  provide  support  during  the  previous  5  years. 
Twenty  offices  provided  responses,  which  were  the  basis  for  the 
review  document. 

The  review  document  stated  that  all  responders  to  the  survey 
except  one  indicated  a  continuing  requirement  for  the  services 
provided  by  IDA.  The  office  that  no  longer  required  IDA  services 
based  its  decision  on  IDA's  poor  performance  on  a  particular 
study.  The  review  document  cited  IDA's  capability  to  maintain  a 
qualified  staff  in  certain  areas  who  were  "attuned”  to  OSD  needs 
as  the  primary  reason  for  renewing  the  contract.  Two  other 
attributes  cited  were  IDA'S  objectivity  and  independence  (IDA 
performed  no  work  for  private-sector  firms  or  the  Military 
Departments) .  The  review  document  also  included  complimentary 
comments  on  the  quality  and  expertise  of  the  IDA  staff,  the  close 
relationship  of  IDA  with  its  sponsors,  and  IDA's  trustworthiness 
with  sensitive  information. 

The  review  document  stated  that  comments  on  the  adequacy  of  IDA's 
management  were  mixed  and  that  IDA'S  management  was  the  only 
significant  performance  criticism  on  certain  projects. 
One  sponsor  said  that  IDA  incurred  costs  that  were  not  authorized 
on  the  project.  Another  mentioned  that  IDA  incurred  costs  of 
$900,000  on  a  particular  project  without  any  useful  results.  The 
office  that  terminated  any  further  use  of  IDA  stated  that  IDA  had 
several  opportunities  during  the  5  years  to  improve  its 
performance,  but  its  performance  had  not  improved.  For  the  most 
significant  project,  "IDA  subcontracted  the  work  and  lost  total 
control  of  the  project."  The  majority  of  the  respondents, 
however,  provided  positive  comments. 

The  sole-source  justification  described  the  services  to  be 
provided  by  IDA  as  follows: 

...task  order  projects  directed  to  individual  issues 
of  urgent,  near-term  and  long-term  interests  in  the 
planning  and  management  of  defense  and  national 
security-related  programs  in  the  following  areas: 
application  of  science  and  technology  in  national 
security  matters;  exploration  of  issues  in  defense 
systems  research  and  development;  computer  and 
software  engineering;  evaluation  of  military  systems 
in  development  or  proposed,  and  of  military  forces 
using  those  systems;  analysis  of  manpower,  readiness 
and  support  issues;  research  into  the  costs  of 
military  systems,  personnel  and  activities;  research 
in  strategy,  military  planning,  international 
security,  and  related  defense  policy  and  management; 
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*  assessment  of  worldwide/  regional  and  local  balances 
of  power  and  stability  and  the  factors  affecting  them 
including  their  technical,  economic,  and  military 
aspects;  operational  evaluation  and  analyses  of 
systems,  forces,  and  military  organizations  in  tests, 
wargames,  field  operations,  and  actual  combat.  ' 

The  justification  stated  that  such  work  could  only  be  performed 
by  IDA  because  of; 

...its  unquestioned  objectivity,  free  from  conflicts 
of  interests  caused  by  the  Services,  commercial  or 
other  involvement;  its  confidentiality  to  protect  very 
sensitive  military  and  intelligence  information  as 
well  as  proprietary  information  from  commercial 
contractors;  its  stable,  interdisciplinary  staff 
Structures  to  meet  the  requirements  of  its  sponsors; 
its  familiarity  with  the  needs  of  its  sponsors;  its 
establishment  of  a  continuing  research  agenda  for  its 
sponsors;  and  its  continuing  high  quality  research 
products  and  advice  provided  to  its  sponsors. 

The  justification  further  stated  that  no  market^  survey  was 
performed  because  of  the  unique  status  and  capability  of  the 
FFRDC  at  IDA.  The  authority  cited  for  the  sole-source  contract 
was  FAR  6.302-3.  The  justification  did  not  mention  that  the 
award  was  publicized  in  the  Commerce  Business  Daily  and  that  a 
number  of  organizations  had  expressed  interest  in  competing  for 
the  contract. 

Synopsis  of  contract  in  Cemmerce  Business  Daily,  On 
August  17,  1988,  DSS-W  published  a  notice  of  the  ^  proposed 
sole-source  award  to  IDA  in  the  Commerce  Business  Daily.  The 
synopsis  stated: 

STUDIES  AND  ANALYSES  in  support  of  the  Office  of  the 
Secretary  of  Defense  Federally  Funded  Research  and 

Development  center  Program  Office - The  supplies  or 

services  required  to  meet  the  agency's  needs  are:  task 
order  projects  directed  to  individual  issues  of 
urgent,  near-term  and  long-term  interests  in  the 
planning  and  management  of  defense  and  national 
security-related  programs  in  the  following  areas: 

Application  of  science  and  technology  in  national 
security  matters;  exploration  of  issues  in  defense 
systems  research  and  development;  computer  and 
software  engineering;  evaluation  of  military  systems 
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in  development  or  proposed,  and  of  military  forces 
using  those  systems;  analysis  of  manpower,  readiness 
and  support  issues;  research  into  the  cost  of  military 
systems,  personnel  and  activities;  research  in 
strategy,  military  planning,  international  security, 
and  related  defense  policy  and  management;  assessment 
of  world-wide,  regional,  and  local  balances  of  power 
and  stability  and  the  factors  affecting  them  including 
their  technical,  economic  and  military  aspects; 
operational  evaluation  and  analyses  of  systems, 
forces,  and  military  organizations  in  tests,  wargames, 
field  operations,  and  actual  combat.  This  synopsis  is 
for  informational  purposes  only.  A  sol  (solicitation] 
is  not  available. 

In  response  to  this  announcement,  the  following  nine  commercial 
organizations  expressed  written  interest  in  competing  for  the 
contract:  BMY  Division  of  Harsco  Corporation;  Price  Waterhouse; 
Acquisition  Dynamics,  Incorporated;  SRS  Technologies;  University 
of  Maryland;  Global  Associates,  Limited;  Epoch  Engineering, 
Incorporated;  COMARCO  International  Business  Services;  and 
KDT  Industries.  In  letters  dated  September  9,  1988,  or 

October  6,  1988,  the  DSS-W  contracting  officer  informed  each  of 

the  first  eight  organizations  listed  above  that; 

The  announcement's  reference  to  studies  and  analyses 
for  the  Office  of  the  Secretary  of  Defense  may  have 
been  misleading.  The  announcement  really  had  to  do 
with  the  renewal  of  a  Federally  Funded  Research  and 
Development  Center.... If  you  think  your  company  is 
qualified  to  perfomm  as  a  Federally  Funded  Research 
and  Development  Center,  you  must  submit  a  capability 
statement  setting  forth  information  that  would  qualify 
you  for  such  effort. 

In  responding  to  the  synopsis,  KDT  Industries,  submitted  a 
four-page  letter  dated  August  23,  1988,  that  described  its 

capabilities,  specific  research  efforts,  affiliations,  and  staff 
capabilities. 

In  a  September  22,  1988,  letter  to  KDT  Industries,  the  Deputy 

Director  for  Acquisition,  DSS-W,  stated: 

The  Director. . .OSD. . .PFRDC  Programs  Office  acts  as  my 
representative  for  two  DoD  contracts  with 
FFRDC's. . . . In  this  capacity  he  does  not  establish  the 
research  requirements  to  be  performed  under  these 
contracts.  The  requirements  for  research  to  be 
performed  by  the  FFRDC's  are  determined  by  individual 
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offices  within  OSD.... Since  PFRDC's  do  not  normally 
perform  work  outside  of'  the  PFRDC  contract,  their 
research  staff  must  be  developed  and  maintained  based 
on  the  specific  needs  of  the  sponsoring 
of f ices. .. .Because  of  the  Government's  “investment"  in 
establishing  the  capability  within  these  FFRDC's  and 
the  continuing  requirement  for  their  services,  it 
would  not  be  cost-effective  to  employ  other  sources  to 
meet  these  requirements. 

Evaluation  of  the  review  and  iustificatlon.  The  review 
document  and  sole-source  justification  were  inadequate.  The 
survey  in  support  of  the  comprehensive  review  was  not  started 
until  July  25,  1988.  Thus,  the  review  document  was  not  started 
until  after  the  sole-source  justification  that  was  dated 
June  9,  1988.  The  review  was  approved  by  the  Deputy  DDR&E 
(Research  and  Advanced  Technology)  on  December  26,  1989,  more 

than  a  year  after  the  contract  award.  The  requirements  for  the 
FFRDC  were  not  clearly  defined,  and  the  review  document  did  not 
adequately  differentiate  between  work  that  could  be  performed  as 
effectively  by  additional  in-house  staff  or  by  contracts  for 
advisory  and  assistance  services.  The  assessment  of  efficiency 
and  effectiveness  in  meeting  sponsor  needs  covered  a  5-year 
timeframe  and  was  not  based  on  any  objective  criteria  or 
measurement  standards.  The  review  document  did  not  discuss 
management  control  procedures  implemented  by  IDA  management  to 
control  costs,  although  comments  were  received  regarding 
unauthorized  and  excessive  costs.  Also,  the  review  document  did 
not  establish  that  the  FAR  criteria  for  establishing  an  FFRDC 
continued  to  be  satisfied  or  that  an  acceptable  sponsoring 
agreement  was  maintained. 

In  summary,  we  concluded  that  the  review  document  and 
justification  did  not  support  the  award  of  a  5-year  sole-source 
contract  with  an  estimated  value  of  $250  million. 

Manaaem^T^fc  Gamments  and  Audit  Response 

Management  rsommenta  -  The  Deputy  DDR&E  recommended  the 
discussion  of  the  Commerce  Business  Daily  synopsis  and  responses 
to  the  synopsis  be  deleted.  He  stated  that  DSS-W  responded 
properly  to  the  contractors  that  expressed  written  interest  in 
competing  for  the  contract  and  none  of  the  organizations 
submitted  a  formal  protest. 

Audit  response.  We  did  not  delete  the  discussion  regarding 
the  cniTiTniarne  Business  Daily  synopsis  because  the  responses 
indicate  competitive  private-sector  firms  were  willing  to 
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perform  studies  and  analyses  in  the  areas  identified  in  the 
synopsis.  The  synopsis  did  not  identify  the  essential  core 
capabilities  that  would  make  the  FFRDC  a  unique  resource  for 
satisfying  special  research  and  development  needs.  The 
DSS-W  responses  dated  September  9,  1988,  and  October  6, 
1988,  did  not  justify  the  noncompetitive  renewal  of  the 
FFRDC  contract  although  the  sole-source  justification  was 
approved  in  June  1988.  Also,  the  DSS-W  response  to 
KDT  Industries  contained  a  statement  that  using  other 
sources  was  not  cost-effective  and  the  basis  for  this 
statement  was  not  supported  by  documentation. 
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Background.  In  April  1942,  the  Navy  organized  the 
Antisubmarine  Warfare  Operations  Research  Group  to  analyze  and 
improve  U.S.  defenses  against  German  submarine  warfare.  The 
Antisubmarine  Warfare  Operations  Research  Group, ^  along  with  the 
Navy's  Operations  Research  and  Operations  Evaluation  Groups,  were 
predecessors  to  the  Center  for  Naval ' Analyses.  Between  1944  and 
1962,  the  mission  of  those  organizations  changed  from 
antisubmarine  warfare  research  to  new  equipment,  tactical 
doctrine,  and  strategic  warfare  analysis  and  evaluation  for  the 
Navy  fleets.  MIT,  which  managed  an  Operations  Evaluation  Group, 
and  IDA,  which  operated  an  Institute  for  Naval  Analyses,  provided 
contract  support  to  the  two  groups. 

In  1962,  the  Navy  combined  the  Operations  Evaluation  Group  and 
the  Institute  for  Naval  Analyses  and  created  the  Center  for  Naval 
Analyses  (CNA) .  CNA  was  designated  a  non-profit  FFRDC  to  be 
managed  by  the  Franklin  Institute  of  Pennsylvania.  Arrangements 
with  MIT  and  IDA  were  terminated.  Between  1962  and  1983,  CNA's 
mission  was  to  conduct  studies  and  analyses  that  would  assist  the 
Navy  in  Naval  application  and  development  decisionmaking. 

In  1983,  because  of  differences  over  the  proper  management  of 
CNA,  the  Navy  decided  to  open  the  CNA  contract  to  competition. 
Several  universities  and  non-profit  research  organizations 
responded  to  the  solicitation.  In  August  1983,  the  Navy  selected 
the  Hudson  Institute  of  New  York  to  manage  CNA,  effective 
October  l,  1983.  Under  this  contract,  CNA's  mission  continued  to 
be  assisting  the  Navy  with  studies  and  analyses. 

In  May  1990,  the  Hudson  Institute  notified  the  Navy  of  its 
intention  to  terminate  the  contract.  The  Navy  decided  to  award  a 
noncompetitive  contract  to  CNA,  Incorporated,  a  new  nonprofit 
"shell"  corporation  formed  specifically  to  operate  CNA.  Contract 
N00014-91-C-0002  with  CNA,  Incorporated,  was  effective  October  1, 
1990,  for  5  years  at  a  total  estimated  cost  of  $240  million. 

Basis  for  sole-source  contract.  The  Deputy  Chief  of  Naval 
Operations  (Navy  Program  Planning)  prepared  a  10-page  review 
document  dated  November  30,  1990,  and  a  sole-source  justification 
dated  July  17,  1990,  for  the  September  28,  1990,  award  of  the 
sole— source  contract  to  CNA.  The  review  document  was  the  same 
format  used  by  the  DDR&E  for  the  IDA  review  document,  and 
included  much  of  the  same  wording.  The  first  four  pages 
discussed  the  background  of  CNA,  its  organization,  and 
sponsorship.  The  remaining  six  pages  discussed  the  results  of 
the  sponsor's  review. 
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In  a  May  29,  1990,  memorandum  the  Deputy  Chief  of  Naval 

Operations  (Navy  Program  Planning)  asked  39  CNA  users  from  the 
past  several  years  to  assess  CNA  based  on  the  five  criteria  in 
FAR  35.017-4.  A  2-page,  12-item  questionnaire  sent  to  each  user 
assisted  in  the  assessment.  The  users  were  asked  for  check-mark 
rating  of  "Strongly  agree,"  "Agree,"  "Disagree,"  "Strongly 
disagree,"  and  "No  opinion."  Questions  related  to  the  depth  and 
breadth  of  technical  expertise  and  the  operational  expertise  of 
the  CNA  staff  and  asked  for  comparisons  to  alternate  analytic 
sources.  The  questionnaire  asked  whether  CNA  work  benefited 
significantly  from  analyst  continuity,  provided  timely 
assistance,  demonstrated  efficient  and  effective  procedures  and  a 
cost-effective  operation  in  meeting  needs,  and  whether  CNA 
adhered  to  policy  guidance  in  OFPP  Letter  84-1.  Also,  users  were 
asked  for  the  percent  of  work  performed  by  CNA  that  could  be  done 
by  other  organizations.  Finally,  users  were  asked,  "If  resources 
were  available,  how  would  you  change  how  your  organization's 
analytic  requirements  are  met?"  The  check-mark  responses  to  this 
question  were  "Increase..."  "Decrease..."  or  "Make  no  change..." 
in  the  number  of  CNA  analysts. 

The  review  document  was  based  on  33  responses  to  the  survey  and 
included  the  following  statements: 

Continuing  Requirement  For  the  Services  Provided  bv  CNA 


All  responders  indicated  a  continuing 
requirement. .. .They  would  either  make  no  change  or 
increase  the  number  of  CNA  analysts  in  order  to 
satisfy  their  analytic  requirements. .. .All  responders 
agreed  that  CNA  possesses  the  depth  and  breadth  of 
technical  expertise  to  meet  their  analytic 
needs.... The  special  relationship  which  has  developed 
between  sponsors  and  the  CNA  staff,  particularly  the 
field  representatives,  has  allowed  CNA  to  be  more 
responsive  to  the  sponsors'  individual  needs  and  has 
provided  the  mechanism  for  timely  products  with  much 
less  risk  than  one  would  expect  from  a  private 
contractor . 

Alternative  Sources 

Most  responders  (90%)  indicated  that  CNA  work  is  more 
useful  in  framing  problems,  in  answering  key 
questions,  and  in  explaining  results  and  interpreting 
output  and  that  CNA  work  is  broader  in  scope  and  more 
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in  depth.  Most  responders  (84%)  indicated  that  CNA 
work  is  more  analytically  rigorous^  more  operationally 
realistic,  more  objective,  and  more  credible;  that  CNA 
work  is  more  timely;  and  that  CNA  work  is  higher  in 
overall  quality. .. .Most  responders  (89%)  indicated 
that  the  majority  of  work  now  being  done  by  CNA  could 
not  be  accomplished  by  contractor  analysts. 

Efficiency  and  Effectiveness  of  CNA 

...almost  all  (94%)  of  the  sponsors  agreed... CNA 
demonstrates  efficient  and  effective  procedures  in 
meeting  the  organization's  analytic  needs.... CNA 
provides  timely  assistance  in  meeting  the 
organizations  analytic  needs. 

Adequacy  of  CNA  Management 

Most  of  the  sponsors  of  CNA  work  stated  that  CNA 
management  was  supportive  to  the  sponsor's  particular 
requirements. .. .CNA  was  well-managed,  cooperative,  and 
responsive.  Responses  from  a  majority  of  sponsors 
(75%)  indicated  that  CNA  management  demonstrates  a 
cost  effective  operation.  The  remainder  had  no 
opinion.  These  responses  are  not  surprising:  for 
commands  outside  Washington,  the  primary  means  of 
visibility  to  CNA  management  is  through  the  field 
program. 

Compliance  With  Criteria  For  Establishing  the  FFRDC 

Responses  from  sponsors  were  limited. . .since  most 
sponsors  would  not  be  aware  of  the  directive  [OFPP 
Letter  84-1],  except  for  this  review  of  CNA.... a 
majority  (81%)  agreed  that,  to  the  best  of  their 
knowledge,  CNA  adheres  to  OFPP  guidance. .. .A  majority 
(72%)  agreed  that  CNA. ..[ identifies J  work  which  is 
inappropriate  for  a  FFRDC  to  perform. 

The  sole-source  justification  described  the  services  to  be 
procured  from  CNA  as  follows: 

The  CNA  provides  the  Navy  and  the  Marine  Corps  with  an 
independent  source  of  analysis  and  evaluation  based  on 
its  unique  access  to  sensitive  data  and  the  hands-on 
exposure  to  fleet  operations  through  its  world-wide 
field  program.  The  CNA  conducts  a  wide  range  of 
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research  that  provides  two  fundamental  services  to  the 
Navy  and  Marine  Corps;  (1)  on-site  analyses  for  the 
fleet  commanders  to  improve  tactics  and  readiness  of 
existing  forces  and  (2)  analyses  for  Navy  and  Marine 
Corps  headquarters  decision  makers  with  responsibility 
in  areas  such  as  weapon  systems,  tactics,  concepts  of 
operations  and  doctrines,  future  naval  policy,  new 
strategic  and  operational  concepts,  major  force 
levels,  warfare  capability  assessment  and  Navy  program 
planning  •  This  type  of  work  can  only  be  performed  by 
the  CNA  because  of:  its  unquestioned  objectivity,  free 
from  conflicts  of  interest  caused  by  the  Navy, 
commercial  or  other  involvement;  its  confidentiality 
to  protect  very  sensitive  military  and  intelligence 
information  as  well  as  proprietary  information  from 
commercial  contractors;  its  stable,  interdisciplinary 
staff  structure  to  meet  requirements  of  the  Navy  and 
Marine  Corps;  its  familiarity  with  the  needs  of  the 
Navy  and  Marine  Corps;  its  establishment  of  a 
continuing  research  agenda  for  the  Navy  and  Marine 
Corps;  and  its  continuing  high  quality  research 
products  and  advice  provided  to  the  Navy  and  Marine 
Corps. 

The  sole-source  justification  included  the  following  statement 
regarding  the  reasonableness  of  the  operating  costs  of  CNA; 

Although  the  Center  for  Naval  Analyses  will 
incorporate  as  a  separate  self  managing  entity,  the 
Contracting  Officer  does  not  envision  any  radical 
change  in  the  CNA's  accounting  system.  The  new  entity 
will  maintain  the  same  management  and  staff  without 
the  outside  layer  of  Hudson  Institute's  management. 

The  offerer  has  submitted  a  cost  proposal  which 
provides  for  direct  labor  years  which  are  within  the 
Congressionally  approved  ceiling. 

The  sole-source  justification  stated  that  a  market  survey  was  not 
performed  and  that  no  other  sources  had  expressed,  in  writing,  an 
interest  in  the  acquisition.  The  sole-source  justification  was 
approved  by  the  Assistant  Secretary  of  the  Navy  (Research, 
Development,  and  Acquisition)  and  the  contract  was  awarded  by  the 
Office  of  Naval  Research. 

Evaluation  of  the  review  and  justification.  The  review 
document  was  not  prepared  and  approved  until  November  30,  1990, 

after  the  sole-source  justification  approval  and  contract  award 
on  September  28,  1990.  The  review  document  did  not  adequately 
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define  future  requirements  and  did  not  address  the 
cost-effectiveness  of  CNA  services  or  why  the  services  could  not 
be  provided  by  a  combination  of  additional  Navy  personnel  and 
private  contractors . 

The  review  document  did  not  provide  an  adequate  rationale  for 
transferring  the  operation  of  CNA  from  the  Hudson  Institute  to 
CNA,  Incorporated,  a  new  non-profit  corporation.  Neither 
document  contains  an  explanation  of  why  the  Navy  did  not 
competitively  solicit  proposals  for  a  new  operator.  CNA's 
operational  efficiency  and  effectiveness  was  assessed  on  the 
basis  of  subjective  ratings  of  whether  CNA  met  the  analytic  needs 
of  the  customer  and  was  timely,  and  did  not  discuss  the 
reliability  of  its  data  collection  and  analysis,  reporting, 
computer  operations,  and  organizational  structure.  The 
assessment  of  efficiency  and  effectiveness  covered  work 
accomplished  by  CNA  when  it  was  operated  by  the  Hudson  Institute. 
Responses  did  not  specify  the  extent  of  work  considered  in  making 
the  assessment.  The  review  document  also  did  not  document  the 
management  controls  implemented  by  CNA  to  assure  a  cost-effective 
operation  and  did  not  define  the  mission  of  CNA  clearly  enough  to 
differentiate  CNA  work  from  work  appropriate  for  in-house  or 
non-PFRDC  contractors.  The  review  document  also  did  not 
adequately  address  compliance  with  the  FAR  criteria  for 
establishing  an  FFRDC  or  for  maintaining  a  sponsoring  agreement. 

We  also  determined  that  the  statistics  in  the  review  document 
regarding  user  consensus  about  CNA  management  demonstrating  a 
cost-effective  operation  were  misleading.  Navy  and  Marine^  Corps 
activities  responding  to  the  survey  were  not  in  a  position  to 
comment  on  the  cost-effectiveness  of  CNA's  operations  because  the 
Navy  centrally  funded  CNA  studies,  and  individual  users  were  not 
provided  an  accounting  of  costs  for  their  projects.  We  concluded 
that  the  justification  did  not  adequately  support  the  award  of  a 
5-year  sole-source  contract  with  an  estimated  cost  of 
$240  million. 

Cost-effectiveness  of  CNA  field  analysts.  CNA  was  the  only 
FFRDC  that  had  field  analysts  assigned  full-time  to  Navy  Commands 
around  the  world.  The  review  and  justification  did  not 
distinguish  how  the  CNA  field  analysts  services  differed  from 
personal  services  since  the  CNA  field  analysts  were  supervised  by 
Navy  personnel.  CNA  used  their  field  program  as  a  training 
ground  for  junior  analysts.  The  field  analysts  performed  duties 
that  included  observing  fleet  exercises,  reconstructing  fleet 
operations  exercises  and  generating  after  action  reports, 
evaluating  fleet  information  acquisition  systems  and  preparing 
quick  analyses,  developing  training  requirements,  teaching,  and 
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attending  training.  We  concluded  that  these  duties  could  be 
performed  by  civilian  personnel  and  junior  Naval  officers. 
During  FY  1991,  the  CNA  field  program  included  41  analysts 
assigned  to  34  Navy  and  Marine  Corps  field  commands  and  cost 
about  $8.3  million.  If  GS/GM-15/5  analysts  had  been  employed  by 
the  Navy  and  Marines  instead  of  the  CNA  analysts,  we  estimate 
that  $3.1  million  of  costs  would  have  been  eliminated  during  FY 
1991.  Over  the  2  remaining  years  of  the  CNA  contract  (FYs  1994 
and  1995),  about  $6.2  million  of  costs  (2  years  at  $3.1  million 
per  year)  can  be  eliminated  by  using  in-house  civilian  personnel 
instead  of  CNA  field  analysts. 

The  following  table  shows  the  basis  for  our  estimate. 

Cost  Comparison  of  CNA  Field  Analysts 
Versus  In-house  Civilian  Personnel 
_ FY  1991 _ 


CNA 

In-house 

Annual 

Actual 

Estimated 

Costs 

Costs^' 

Costs 

Eliminated 

Salaries 

$2,266 

(thousands) 

$2,864^/ 

($598) 

Allowances 

504^/ 

265^/ 

504 

0 

Overtime /At- sea  Pay 

265  , 
849^/ 

0 

Benefits 

1,170 

321 

Other  Overhead,  General 
Administrative  Costs 

& 

3,391^' 

0 

698^/ 

3,391 

Other  Direct  Charges 

698 

0 

Total 

$8.294 

$5.180 

$3,1-14. 

^/fiased  on  data  provided  by  CNA. 

^/salary  of  41  GS/GM-15/5s  at  $69,863  per  year  (1991)  was  used 
for  comparison.  A  manpower  review  may  determine  that  the 
equivalent  grade  level  of  the  work  done  by  the  field  analysts 
is  lower  than  GS/GM-15/5. 

-^Includes  incentive,  site,  and  transportation  allowances  at 
certain  locations  ($504,000)  and  extended  duty/ sea-pay 
($265,000);  CNA  actual  and  in-house  estimated  costs  assumed 
equal . 

-'Fringe  benefits  estimated  at  29.65  percent  based  on  Office  of 
Management  and  Budget  Circular  A-76. 

^'Includes  facility,  utility,  equipment,  etc.,  costs  of  CNA 
headquarters.  Field  representatives  are  supported  by  local 
commands  and  no  additional  in-house  costs  are  assumed  in 
supporting  civilian  employees. 

-'Includes  $509,000  relocation  and  $156,000  travel;  CNA  actual 
and  in-house  estimated  costs  assumed  equal. 
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Management  Coimmeniia  and  Audit  Response 

Management  comments .  The  Navy  nonconcurred  with  the 
statement,  ”The  review  document  also  did  not  provide  an  adequate 
rationale  for  transferring  the  operation  of  CNA  from  the  Hudson 
Institute  to  CNA,  Incorporated. . . . [or]  contain  a  formal 
explanation  of  why  the  Navy  did  not  competitively  solicit 
proposals  for  a  new  operator.”  The  Navy  stated  that 
FAR  35.017-2 (h)  provides  for  autonomous  organizational  structures 
such  as  the  structure  at  CNA.  The  Navy  added  that  once  the  Navy 
determined  that  CNA  could  effectively  manage  itself,  imposing 
another  parent  organization  to  manage  CNA  was  not  necessary  and 
would  only  impose  an  unnecessary  layer  of  management  and  cost. 

Audit  response.  The  comprehensive  review  for  CNA  did  not 
adequately  explain  why  the  Navy  helped  establish  a 
corporation  for  the  sole  purpose  of  operating  a  Navy  FFRDC. 
The  Senate  Subcommittee  on  Oversight  of  Government 
Management  report,  "Inadequate  Federal  Oversight  of 
Federally  Funded  Research  and  Development  Centers,"  July  8, 
1992,  also  considered  the  Navy's  rationale  to  be 
insufficient  because  the  rationale  raises  questions  about 
FFRDC  independence  and  accountability.  The  subcoimittee 
recommended  that  OFPP  study  the  costs  and  benefits  of 
permitting  "shell  corporations"  such  as  CNA  to  operate 
FFRDCs.  OFPP  agreed  to  perform  a  study. 

Managremeni:  eommewfca.  The  Navy  stated  that  there  is 
"ABSOLUTELY  NO  REQUIREMENT"  that  the  basic  purpose  and  mission  of 
the  FFRDC  must  be  contained  in  the  comprehensive  review 
documents . 

Audit  response.  We  disagree.  FAR  35.017-2(f), 
"Establishing  or  Changing  an  FFRDC,"  requires  the  sponsor  to 
ensure  that  the  basic  purpose  and  mission  of  an  FFRDC  is 
stated  clearly  enough  to  enable  differentiation  between  work 
that  should  be  performed  by  the  FFRDC  and  work  that  should 
be  performed  by  non-FFRDCs.  In  addition,  FAR  35.op-4(c)  (5) 
requires  that  comprehensive  review  documents  include  a 
determination  that  the  criteria  for  establishing  the  FFRDC 
continue  to  be  satisfied.  To  fully  satisfy  these 
requirements,  the  comprehensive  review  document  must 
identify  the  principal  purpose  and  mission  of  the  FFRDC. 

M»Ttageinef^4-  riftnmiftTita.  The  Navy  partially  concurred  that  Navy 
and  Marine  Corps  activities  did  not  have  individual  project  cost 
data  when  the  comprehensive  review  survey  was  conducted.  The 
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Navy  stated  that  the  CNA  Policy  Council  is  periodically  briefed 
regarding  funds  for  the  CNA  research  program.  The  CNA  Policy 
Council  is  satisfied  that  CNA  provides  valuable  services  at 
reasonable  costs.  The  Navy  added  that  users  will  have  individual 
project  cost  data  during  future  comprehensive  reviews. 

Audit  response.  The  Navy's  plans  to  provide  users  project 
cost  reports  during  future  comprehensive  reviews  is  an 
improvement.  Prior  to  the  audit,  the  prevailing  attitude  of 
the  CNA  Policy  Council  regarding  the  cost  reasonableness  of 
CNA  services  was  based  on  intuition  rather  than  performance 
measures  that  documented  the  cost  relationships  between 
outputs  produced  and  resources  used  to  produce  them. 
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Background .  The  Logistics  Management  Institute  (LMI)  was 
incorporated  in  Delaware  as  a  non-profit  organization  in  October 
1961.  The  certificate  of  incorporation  stated  the  objectives  and 
purposes  of  LMI  were; 

...to  foster  and  enco.urage  the  advancement  of 
knowledge  concerning  logistics  in  all  of  its  aspects 
and  to  perform,  engage  in,  and  procure  research, 
development,  engineering  and  advisory  services 
exclusively  to  or  for  the  O.S.  Government  or  any 
department  or  agency  thereof  or  any  other  government 
or  governmental  unit  or  any  nonprofit  corporation  or 
other  organization  organized  and  operated  exclusively 
for  scientific,  educational,  or  charitable  purposes. 

Shortly  after  its  incorporation,  the  Secretary  of  Defense  awarded 
LMI  its  first  5-year  contract  for  studies  directed  at  improving 
the  business  management  of  DoD. 

LMI  was  designated  an  FFRDC  in  1984  when  its  contract  for  studies 
and  analyses  was  renewed  because  all  of  the  requirements  of  OFPP 
Letter  84-1  were  satisfied.  From  its  inception,  LMI  has 
performed  studies  of  interest  to  OSD  and  other  DoD  Components, 
un  has  also  performed  studies  requested  by  the  Agency  for 
International  Development,  the  Department  of  Health  and  Human 
Services,  the  Department  of  State,  the  National  Aeronautics  and 
Space  Administration,  the  National  Institute  of  Health,  the 
National  Park  Service,  the  Public  Heal'^h  Service,  and  the  Coast 
Guard . 

Basis  for  1990  sole-source  contract*  The  Office  of  the 
Assistant  Secretary  of  Defense  (Production  and  Logistics) 
prepared  a  review  document  dated  July  19,  1989,  and  sole— source 
justification  dated  November  16,  1989,  for  the  November  20,  1989, 
contract  awarded  by  DSS-W.  The  effective  date  of  the  contract 
was  October  1,  1989.  The  two-page  review  document  stated; 

...the  review. . .included  the  separate  Military 
Services,  the  Joint  Staff,  other  Federal  Agencies  and 
the  Staff  of  the  OASD  (P&L) .  Without  exception  the 
responses  are  overwhelmingly  supportive  of  the 
continued  need  for  and  use  of  LMI  as  a  primary 
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research  center  for  the  Department  of  Defense.  The 
evaluation  format  covered  the  following  areas  of 
performance: 

o  LMI  Participation  &  Assignments 
o  Program  Objectives 
o  Working  Relationships 
o  Project  Productivity 
o  Quality  of  Products 
o  Project  Staffing 
o  Summary  Appraisal 

Ninety-nine  percent  of  the  responses  rated  LMI  as 
ALWAYS  performing  at  the  highest  level  in  the 
categories  listed  above.  It  is  requested  that  a  new 
five-year  contract  be  prepared  and  negotiated  with  the 
Logistics  Management  Institute  through  Fiscal  Year 
1994. 

The  two— page  sole-source  renegotiation  justification  stated  the 
services  to  be  procured  were: 

...task  order  directed  research  and  analyses  in  the 
areas  of  logistics  and  weapon  systems  acquisition  to: 

(1)  reduce  costs  and  increase  the  effectiveness  of 
military  procurement,  materiel  management,  logistics 
and  manpower  support  activities;  <2)  formulate  and 
recommend  changes  in  DoD  policy  relating  to 
acquisition  and  support  of  weapons  systems  and  other 
defense  resources  requirements;  (3)  develop 
mathematical  models  and  other  management  tools  for  the 
evaluation  of  logistics  and  manpower  plans  and 
materiel  requirements;  and  (4)  appraise  the  materiel 
readiness  of  the  Armed  Forces. 

The  justification  cited  FAR  6.302-3  as  the  authority  for  the 
sole-source  procurement.  No  market  survey  was  performed. 

Synopsis  of  contract  in  Commerce  Business  Daily*  DSS-W 
published  a  notice  of  the  proposed  sole-source  award  to  LMI  in 
the  Commerce  Business  Daily  on  August  8,  1989.  The  synopsis 

stated: 


CONTINUING  RESEARCH  IN  SUPPORT  OF  OSD... The  Dept  of 
Defense  plans  to  award  a  sole-source  follow-on 
contract  to  the  Federally  Funded  Research  and 
Development  Center  at  the  Logistics  Management 
Institute,  Bethesda,  MD  for  continuation  of  research 
currently  sponsored  under  Contr  [Contract]  No.  MDA903- 
85-C-0139  by  various  elements  of  the  Office  of  the 
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Secretary  of  Defense,  other  Defense  Agencies,  and  the 
Military  Depts  [Departments].  The  research  addresses 
a  broad  range  of  policy  formulation,  assessment  and 
implementation  issues  in  acquisition,  logistics, 
manpower,  force  readiness,  reserve  affaire,  and 
related  areas.  See  Note  22.  [Note  22  stated:]  The 
proposed  contract  action  is  for  supplies  or  services 
for  which  the  Government  intends  to  solicit  and 
negotiate  with  only  one  source  under  authority  of 
FAR  6.302.  Interested  persons  may  identify  their 
interest  and  capability  to  respond  to  the  requirement 
or  submit  proposals.  This  notice  of  intent  is  not  a 
request  for  competitive  proposals ...  .A  deteimnination 
by  the  Government  not  to  compete  this  proposed 
contract  based  upon  responses  to  this  notice  is  solely 
within  the  discretion  of  the  government.  Information 
received  will  normally  be  considered  solely  for  the 
purpose  of  determining  whether  to  conduct  a 
competitive  procurement. 

DSS-W  received  no  responses  to  this  notice  in  the  Commerce 
Business  Daily. 

Evaluation  of  review  and  •justification*  The  review  document 
and  the  sole-source  justification  did  not  demonstrate  that  Ull 
conformed  to  the  requirements  for  an  FFRDC.  The  periodic  review 
document  was  based  on  10  responses  to  a  survey  questionnaire  that 
contained  33  questions  with  ratings  of  "always  or  exceptional," 
"usually  or  acceptable,"  and  "rarely  or  unsatisfactory."  The 
questions  did  not  provide  for  an  adequate  assessment  of  any  of 
the  five  criteria  in  FAR  35.017-4  for  the  comprehensive  review. 
None  of  the  questions  addressed  whether  and  at  what  levels  the 
activities  continued  to  have  special  studies  requirements  to  be 
performed  by  LMI  or  the  feasibility  of  alternative  contractor  or 
Government  sources  to  perform  these  requirements.  The  survey 
questions  addressed  LMI's  performance  in  conducting  studies  but 
were  based  on  subjective  criteria.  The  responses  did  not 
identify  the  periods  of  time  or  the  projects  that  were  the  basis 
for  the  ratings.  Accordingly,  we  believe  that  the  surveys  were 
not  an  adequate  assessment  of  LMI's  efficiency  and  effectiveness. 
The  survey  questions  did  not  address  the  existence  or  adequacy  of 
LMI  management  controls  to  ensure  a  cost-effective  operation,  the 
overall  compliance  of  LMI  with  the  criteria  for  its  establishment 
as  an  FFRDC,  or  the  adequacy  of  the  sponsoring  agreement.  The 
review  document  did  not  clearly  differentiate  between  work 
appropriate  for  LMI  and  work  that  could  be  performed  in-house  or 
through  advisory  and  assistance  contracts. 
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The  sole-source  justification  was  based  on  an  inadequate  review 
by  the  primary  sponsor  and  provided  no  details  on  the  level  of 
effort  required.  The  justification  was  inadequate  for  the^  award 
of  a  5-year  sole-source  contract  estimated  to  cost  $80  million. 
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Background.  The  Aerospace  Corporation  (Aerospace)  was 

incorporated  in  the  State  of  California  as  a  nonprofit 

organization  in  June  1960.  Aerospace  was  established 

specifically  to  provide  systems  engineering  and  integration 

support  for  the  development  of  the  ballistic  missile  and  space 
programs.  Before  Aerospace's  establishment,  the  Air  Force  had 
contracted  with  Space  Technology  Laboratories,  a  wholly  owned 
subsidiary  of  Thompson  Ramo  Wooldridge,  Incorporated,  for  systems 
engineering  and  technical  support  on  its  intercontinental 
ballistic  missile  and  space  programs  since  the  early  1950s.  This 
arrangement  led  to  conflicts  of  interest,  and  a  1959  report  by 
the  House  Subcommittee  on  Military  Operations,  Committee  on 
Government  Operations,  noted  that: 

...the  Air  Force  must  retain  or  acquire. . .the  vital 
technical  and  managerial  resources  [Space  Technology 
Laboratories  provides] ... .Complete  objectivity  on  the 
part  of  the  contracting  organization  is,  as  it  should 
be,  the  first  and  foremost  concern  of  the  Air  Force. 

The  value  of  such  an  organization  rests  on  its 
disinterested  position;  the  advice  it  gives  should  be 
based  exclusively  on  the  best  interests  of  the 
Government. 

The  report  called  for  recasting  the  existing  arrangement: 

...into  a  nonprofit  institution  akin  to  the  RAND  Corp. 
and  other  private  and  university-sponsored 
organizations  which  serve  the  military  departments  and 
other  agencies  of  the  Federal  Government  on  a  stable 
and  continuing  basis.  Government  relationships  with 
nonprofit  organizations  also  pose  problems,  but  they 
are  less  important  than  the  benefits  received  and 
certainly  less  crucial  than  those  posed  by  [the  then- 
existing  arrangement]. 


Aerospace  was  originally  staffed  with  people  who  transferred  from 
Space  Technology  Laboratories.  The  original  Aerospace  contract 
and  all  successor  contracts  were  sole-source  awards.  In  addition 
to  the  Air  Force  taskings,  Aerospace  has  worked  on  taskings  from 
the  Strategic  Defense  Initiative  Organization  (now  Ballistic 
Missile  Defense  Organization),  the  Army,  the  Navy,  the  National 
Aeronautics  and  Space  Administration,  the  Department  of  Commerce, 
the  Department  of  Energy,  and  the  Environmental  Protection 
Agency . 

Basis  for  1988  contract.  The  Space  Systems  Division  (now 
Space  and  Missile  Systems  Center) ,  Air  Force  Systems  Command  (now 
Air  Force  Materiel  Command)  prepared  an  acquisition  plan  dated 
March  10,  1988,  and  a  sole-source  justification  dated  April  12, 
1988,  to  justify  the  September  22,  1988,  renewal  of  the  contract 
with  Aerospace.  The  Space  Systems  Division  did  not  prepare  a 
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comprehensive  review  that  addressed  the  periodic  review  criteria 
of  OFPP  Letter  84-1  or  FAR  35.017-4.  The  Acquisition  Strategy 
Plan  (the  Plan)  was  a  12-page  document  that  discussed  the 
services  to  be  provided,  the  applicable  conditions,  the  cost,  the 
capability,  the  delivery  requirements,  the  trade-offs,  the  risks, 
and  the  acquisition  plan.  The  Plan  identified  the  following 
categories  of  effort  to  be  provided  by  Aerospace:  plans  and 
systems  architecture;  general  systems  engineering  and 
integration;  technical  review;  selected  research,  development, 
test,  and  evaluation;  acquisition  support;  engineering  methods; 
mission-oriented  investigation  and  experimentation;  and  foreign 
technology  support.  The  categories  were  not  described  in  detail, 
such  as  specific  programs  skills  to  be  provided  or  levels  of 
effort. 

The  Plan  provided  a  funding  profile  for  the  5  years  covered  by 
the  follow-on  contract.  In  regard  to  capability,  the  Plan  stated 
that  Aerospace  could  obtain  technology  from  Government 
laboratories  and  universities  as  an  FFRDC,  and  discussed 
Aerospace's  background  in  Air  Force  space  programs  and  the 
security  clearance  for  its  facility.  The  Plan  stated  that 
Aerospace's  risk  of  not  meeting  the  planned  schedule  was 
considered  low  because  Aerospace  had  no  major  subcontractors,  was 
not  a  manufacturer,  and  produced  data  reports  as  its  only 
deliverable  item. 

The  sole-source  justification  was  a  nine-page  "class 
justification  and  approval"  document  and  was  signed  by 
seven  officials  at  the  Space  Systems  Division:  the  contracting 
officer,  a  legal  counsel,  the  program  manager,  a  small  and 
disadvantaged  business  utilization  specialist,  the  competition 
advocate,  the  Director  of  Competition  Management,  and  the 
Executive  for  Small  Business.  The  justification  was  also 
approved  by  the  Assistant  Secretary  of  the  Air  Force 
(Acquisition).  The  justification  discussed  the  authority  for 
recommending  a  noncompetitive  contract,  the  qualifications  of 
Aerospace,  a  general  description  of  the  services  to  be  procured, 
and  the  efforts  to  obtain  competition.  The  justification  stated 
that  the  Air  Force  had  not  published  a  sources-sought  synopsis  in 
the  Commerce  Business  Daily  or  performed  a  market  survey  to 
ensure  that  no  other  private-sector  firms  could  perform  the 
required  services.  The  justification  stated  that 

FAR  5.202(a) (10)  allowed  the  contracting  officer  to  waive  these 
requirements  when  the  contract  action  is  authorized  by 
FAR  6.302-3.  The  justification  further  stated: 

...It  is  difficult  to  estimate  how  long  it  would  take 
for  another  contractor  to  develop  the  necessary 
capabilities  to  assume  the  Aerospace  role. ♦••For 
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example,  the  single  greatest  responsibility  of 
Aerospace  is  the  independent  certification  of 
readiness  for  launch  of  spacecraft  and  their  launch 
vehicles.  Executing  this  responsibility  requires 
continuing  and  intensive  work  from  program  initiation 
through  termination  of  space  operations.  It  would  be 
highly  unlikely  that  a  contractor  without  the 
corporate  memory  and  talent  of  Aerospace  could  step  in 
and  perform  this  function. 

The  justification  and  approval  document  stated  that  the 
anticipated  cost  of  the  follow-on  contract  would  be  fair  and 
reasonable  based  on  an  analysis  of  the  actual  cost  of  the  current 
contract  effort  versus  the  proposed  cost,  and  cost_  and  price 
analysis  performed  by  the  Defense  Contract  Audit  Agency, 

administrative  contracting  officer,  price  analyst,  and  other  in- 
house  personnel. 

Evaluation  of  the  acquisition  plan  and  justification.  The 

acquisition  plan  did  not  meet  the  criteria  for  the  required 

comprehensive  review.  The  acquisition  plan  and  justification 
documents  did  not  adequately  define  specific  programs  to  be 
supported  and  the  work  requirements  to  be  tasked  to  Aerospace. 
Neither  the  plan  nor  the  justification  contained  convincing 
evidence  that  private-sector  firms  or  additional  Government 

personnel  could  not  perform  part  or  all  of  the  proposed 

requirements  at  less  cost  and  more  effectively. 

The  documents  also  did  not  discuss  the  operational  efficiency  and 
effectiveness  of  the  FFRDC  in  meeting  needs,  including  such  areas 
as  the  FFRDC's  organizational  structure,  computer  operations, 
reliability  of  data  production  methods,  and  employee 
productivity.  The  justification  and  the  acquisition  plan  did  not 
discuss  the  controls  established  by  the  FFRDC  to  ensure  a 
cost-effective  operation.  Audits  of  contractor  proposals  and 
incurred  costs  by  the  Defense  Contract  Audit  Agency  examine  the 
reasonableness  of  costs,  but  do  not  ensure  the  cost  effectiveness 
of  work  performed  by  FFRDCs.  Further,  the  documents  did  not 
discuss  whether  the  FAR  criteria  for  establishing  the  FFRDC 
continued  to  be  satisfied  or  whether  the  sponsoring  agreement 
complied  with  FAR  35.017-1.  The  justification  did  not  adequately 
support  the  sole-source  award  of  a  5-year  contract  with  estimated 
total  costs  of  more  than  $2.2  billion. 
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APPENDIX  M  -  DETAILS  ON  CONTRACTS  WITH  MITRE  CORPORATION  C^I 
DIVISION 

Rackcf  round.  MITRE  Corporation  (MITRE)  is  a  nonprofit 

corporation  that  was  incorporated  in  Delaware  in  July  1958  to; 

...engage  in,  assist  and  contribute  to  the  support  of 
scientific  activities  and  projects  for,  and  to 
perform,  engage  in  and  procure  research,  development, 
engineering  and  advisory  services  to  or  for,  the 
United  states  Government  or  any  department  or  agency 
thereof. 

MITRE  was  established  to  provide  systems  engineering  support  to 
the  Air  Force  on  the  Semiautomatic  Ground  Environment  Air  Defense 
System,  the  first  large  computer-based  command  and  control 
system.  Lincoln  Laboratory,  which  was  operated  by  MIT,  had 
worked  on  design  of  the  Semiautomatic  Ground  Environment  Air 
Defense  System  from  inception  of  the  program  in  1951,  but  MIT 
officials  believed  that  a  long-term  systems  engineering  effort  as 
envisioned  by  Air  Force  was  not  appropriate  for  a  university 
laboratory.  However,  to  assist  the  Air  Force,  officials  of  MIT 
proposed  that  Lincoln  Laboratory  provide  interim  support  and 
offered  to  support  the  formation  of  a  nonprofit  corporation  to 
take  over  the  work  on  a  permanent  basis.  MIT  would  transfer  the 
work  to  the  new  company  on  subcontract,  and  encourage  the 
transfer  of  Lincoln  people  engaged  in  the  effort.  After  the 
initial  period,  the  new  company  would  become  prime  contractor  for 
the  work.  The  Air  Force  accepted  the  proposal,  and  MITRE  was 
established.  Most  of  the  people  involved  were  transferred  to 
MITRE  on  January  1,  1959. 

Today  the  MITRE  C^I  Division  provides  general  systems  engineer¬ 
ing,  engineering  support,  and  systems  integration  support  to  the 
Air  Force,  the  Army,  the  Navy,  the  Marine  Corps,  the  Joint  Chiefs 
of  Staff,  the  Unified  and  Specified  Commands,  the  OSD,  and  the 
Defense  Agencies.  MITRE  also  provides  support  to  other  Federal 
agencies  and  foreign  governments  through  its  Center  for  Civil 
Systems  and  its  Center  for  Advanced  Aviation  System  Development, 
a  separate  FFRDC  sponsored  by  the  Federal  Aviation 
Administration.  A  1968  amendment  to  the  Articles  of 

Incorporation  deleted  the  restriction  that  all  of  MITRE'S  work 
had  to  be  for  a  department  or  agency  of  the  Federal  Government. 
The  amendment  permits  MITRE  to  enter  into  contracts  with  any 
individual,  firm,  association,  corporation,  or  entity. 
Throughout  MITRE'S  history,  none  of  the  contracts  for  support  of 
DoD  were  ever  awarded  competitively. 

Basis  for  1988  contract.  The  Electronic  Systems  Division 
(now  Electronics  Systems  Center) ,  Air  Force  Systems  Command  (now 
Air  Force  Materiel  Command) ,  prepared  an  undated  review  document 
and  a  sole-source  justification  document  dated  June  23,  1988,  for 
the  October  21,  1988,  award  of  the  contract  for  the  FFRDC  for  an 
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additional  5  years.  The  review  document  was  prepared 

specifically  to  satisfy  the  requirement  of  FAR  35. 017-4 •  For  the 
examination  of  technical  needs  and  mission  requirements,  the 
review  document  stated  that  a  May  1988  review  by  the  Assistant 
Secretary  of  the  Air  Force  (Acquisition)  (SAF/AQ  Federal  Contract 
Research  Center  Review)  includes  a  statement  of  needs  and  current 
and  outyear  efforts.  Regarding  the  consideration  of  alternative 
sources  to  meet  sponsor  needs,  the  review  document  stated: 

The  SAF/AQ  FCRC  Review,  May  1988. .. includes  an 
evaluation  of  in-house  organic  capabilities  and 
support  contractor  capabilities  for  meeting  needs. 

This  office  [Directorate  of  Engineering  and  Program 
Management,  Contract  and  Support  Management]  screens 
each  TO&P  [Technical  Objectives  and  Plans]  to  assure 
work  is  appropriate  for  MITRE,  that  it  can  not  be 
accomplished  by  in-house  or  support  contractors,  and 
that  MITRE  is  the  sole-source  for  the  support 
proposed. 

All  new  work  proposed  for  MITRE  requires  a  Sole  Source 
Certificate,  signed  by  a  General  Officer  or  equivalent 
(SES)  certifying  that  only  MITRE  can  satisfy  the 
Government ' s  requirement . . . 

Program  Start  Reviews  chaired  by  ESD  [Electronic 
Systems  Division]  Commander  or  Vice  Commander  reviews 
all  new  programs  over  $5  million,  and  reviews  the 
proposed  mix  of  support  to  accomplish  the  program, 
including  MITRE,  in-house,  and  support  contractors. 

The  Justification  Review  Document  (JRD),  approved  by 
SAF/AQ,  includes  justification  for  exception  to  the 
Competition  in  Contracting  Act  and  concludes  that  only 
MITRE  can  meet  our  needs  for  the  work  contemplated. 

Concerning  efficiency  and  effectiveness,  the  review  document 
stated  each  MITRE  project  is  reviewed  annually.  The  review 
document  stated  that  continuing  reviews  by  a  resident 
administrative  contracting  officer  and  by  the  Defense  Contract 
Audit  Agency  ensured  a  cost-effective  FFRDC  operation.  The 
review  also  stated  that  the  contract  constituted  the  sponsoring 
agreement  and  that  the  Secretary  of  the  Air  Force  (Acquisition) 
review  and  approval  of  the  sole-source  justification  constituted 
the  required  determinations  that  the  criteria  for  establishing 
the  FFRDC  continued  to  be  satisfied. 
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Eight  Electronic  Systems  Division  officials  signed  the  10-page 
sole-source  justification:  the  contracting  officer,  the  legal 
counsel,  the  program  manager,  the  ^  small  and  disadvantaged 
business  utilization  specialist,  the  Director  of  Contracting,  the 
Executive  for  Small  Business,  the  competition  advocate,^  and  the 
person  who  prepared  the  statement  of  work.  The  justification 
document  stated  the  services  to  be  obtained  from  MITRE  were: 

.♦•general  systems  engineering,  systems  research  and 
planning,  research  and  experimentation,  and  task 
engineering  in  the  field  of  command,  control, 
communications  and  intelligence  (Cl)  systems,  and 
research  in  the  field  of  physics  and  related  sciences, 
and  excess  computer  time,  for  the  Department  of 
Defense,  and  other  Government  agencies  as  approved. 

The  support  falls  into  the  following  research  and 
development  categories;  research,  exploratory 
development,  advanced  development,  engineering 
development,  and  operational  systems  development, 
depending  on  the  program.  The  majority  of  the  work 
falls  in  the  operational  systems  development  category. 

The  services  were  further  described  in  a  statement  of  work 
included  as  an  attachment  to  the  justification.  The  statement  of 
work  said: 


The  preferred  role  for  MITRE  in  systems  acquisition 
will  be  that  of  general  systems  engineering  and 
integration.  As  the  general  systems  engineer  for  a 
specific  ESD  or  other  DoD  agency  system  acquisition 
program,  the  MITRE  Corporation  is  responsible  for 
taking  initiative  in  providing  direct  technical 
support  to  the  Systems  Program  Office. .. .MITRE 's 
responsibilities  deal  with  overall  systems  and  program 
definition;  specification  of  technical  performance 
requirements;  integration  within  the  system  and  with 
associated  systems;  analysis  and  verification  of 
systems  and  subsystem  design;  design  compromises  and 
trade-offs;  definition  of  interfaces;  review  of 
hardware  and  software;  specification  and  review  of 
tests  and  evaluation  of  test  data;  appraisal  of 
contractors'  technical  performance;  and  development  of 
solutions  to  problems  and  technical  alternatives  for 
reduced  program  risk. 

The  justification  also  stated: 

...there  is  ,  very  little  technical  or  schedule  risk 
associated  with  their  role.  Risks  associated  with  the 
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programs  supported  by  MITRE  rest  with  the  prime 
contractors  who  actually  build  and  test  the  hardware* 


The  MITRE  Corporation  has  provided  this  support  since 
1958  when  it  was  formed  as  a  not-for-profit 
corporation  to  fill  the  needs  for  system  engineering 
support  to  the  Air  Force  and  other  DoD  agencies  in  the 
field  of  command,  control,  and  communications,  and 
intelligence  systems* .. .The  proposed  contract  is  for 
continuation  of  this  support*.* 

The  justification  cited  FAR  6.302-3  as  the  authority  for  the 
sole-source  contract  and  stated  that  a  synopsis  was  not  published 
in  the  Commerce  Business  Daily  because  FAR  5.202(a) (1)  permits 
the  contracting  officer  to  waive  the  requirement.  The 

justification  further  stated  that  a  market  survey  was  "Not 
Applicable"  and; 

Further  efforts  were  not  pursued  because  only  8°^® 
source  work  can  be  asslofned  to  MITRE  Corporation  C  I 
Federal  Contract  Research  Center. 

The  justification  also  stated  that  the  anticipated  cost  of  the 
work  would  be  fair  and  reasonable  because; 

Resident  DCAA  [Defense  Contract  Audit  Agency]  auditors 
and  a  resident  Administrative  Contracting  Officer 
provide,  on  a  full-time  basis,  the  surveillance 
required  to  monitor  a  cost-plus-fixed-fee  contract. 


Evaluation  of  the  review  document  and  justification  for  the 
1988  contract.  The  two-page,  undated  periodic  review  document, 
which  should  have  been  the  basis  for  the  sole-source 
justification,  was  prepared  after  the  sole-source  justification 
document .  We  determined  that  the  periodic  review  document  was 
prepared  at  some  indeterminable  time^  after  the  sole-source 
justification  document  because  the  periodic  review  referred  to 
the  May  23,  1988,  sole-source  justification.  The  periodic  review 
document  did  not  adequately  address  any  of  the  FAR  criteria  for 
the  comprehensive  review.  The  periodic  review  document 
quantified  staffing  and  budgetary  requirements  for  FYs  1989 
through  1990,  but  did  not  address  the  special  skills  or 
capabilities  necessary  to  accomplish  the  retirements.  The 
periodic  review  document  did  not  provide  sufficient  justification 
that  private-sector  firms  could  not  provide  part  or  all  of  the 
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requirements  or  that  part  or  all  of  the  work  could  not  be 
performed  at  less  cost  and  more  effectively  with  additional 
Government  personnel. 

Criteria  for  determining  appropriate  circumstances  for  assigning 
work  to  MITRE  were  included  in  Electronic  Systems  Division 
Regulation  80-1,  “Utilization  of  MITRE  Support,”  April  21,  1989. 
However,  the  regulation  does  not  specify  how  to  apply  the 
criteria;  as  the  criteria  are  written,  any  work  could  conceivably 
be  found  appropriate  for  MITRE.  Some  of  the  criteria  are; 

o  need  for  extensive  background  information, 

o  access  to  Air  Force  planning  data, 

o  need  for  outstanding  specialists  in  specific  fields, 
o  need  for  diversified  skills, 
o  continuity  of  effort,  and 
o  need  for  fast  response. 

Although  the  periodic  review  assessed  operational  efficiency  and 
effectiveness,  the  review  was  not  based  on  objective  criteria. 
The  review  asked  for  ratings  of  “ALWAYS” ,  “USUALLY” ,  or  “RARELY” 
to  32  questions  that  addressed  MITRE  C^I  Division  participation 
and  assignments,  program  objectives,  working  relationships, 
project  productivity.,  quality  of  products,  and  project  staffing. 
The  review  also  asked  for  ratings  of  “Exceptional,”  “Acceptable,” 
or  “Unsatisfactory”  to  three  summary  appraisal  questions.  The 
annual  review  stated  the  MITRE  C^I  Division  performance  on 

65.4  percent  of  the  programs  evaluated  was  exceptional, 

33.5  percent  was  acceptable,  and  1.1  percent  was  less  than 
acceptable.  However,  the  ratings  were  not  based  on  any  objective 
criteria.  The  review  also  did  not  address  controls  established 
by  the  FFRDC  to  ensure  a  cost-effective  operation.  The^  review 
did  not  adequately  address  the  continued  satisfaction  or 
compliance  with  the  OFPP  or  FAR  criteria  for  establishing  the 
FFRDC  or  maintaining  a  sponsoring  agreement.  Overall,  the 
justification  did  not  adequately  support  the  award  of  a  5-year 
sole-source  contract  estimated  to  cost  more  than  $2.2  billion. 

1990  realignment.  The  Air  Force  was  the  primary  sponsor  of 
MITRE  from  its  inception  in  1958  until  1990.  On  February  21, 
1990,  the  Under  Secretary  of  Defense  for  Acquisition  (the  Under 
Secretary)  approved  the  realignment  of  the  MITRE  FFRDC  into 
two  divisions  because  of  increasing  amounts  of  non-Air-Force- 
sponsored  support  being  requested  from  and  provided  by  MITRE. 
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The  Assistant  Secretary  of  Defense  (C^I)  became  the  primary 
sponsor  of  the  MITRE  FFRDC.  The  Air  Force,  under  the  existing 
contract,  became  the  sponsor  of  the  MITRE  Bedford  (Massachusetts) 
C^I  Division  and  the  Army  became  the  sponsor  of  the  MITRE 
Washington  C^I  Division  in  McLean,  Virginia.  The  Under  Secretary 
directed  the  formation  of  a  Users  Committee  for  the  MITRE 
Washington  C^I  Division  to  be  jointly  chaired  by  the  Assistant 
Secretary  of  the  Army  (Research,  Development,  and  Technology)  and 
the  Assistant  Secretary  of  Defense  (C^I) .  The  February  21,  1990, 
direction  by  the  Under  Secretary  was  implemented  through  a  May  9, 
1990,  memorandum  of  agreement  and  a  February  11,  1991,  sponsoring 
agreement . 

Before  implementing  the  sponsoring  agreement,  a  staff  assistant 
in  the  Office  of  the  Assistant  Secretary  of  Defense  (C-^I) , 
prepared  a  February  7,  1991,  memorandum,  "Review  of  DoD's 
Continued  Need  for  MITRE "s  C^I  FFRDC",  to  satisfy  the 
FAR  35.017-4  requirement  for  the  comprehensive  review.  The 
memorandum  stated: 


The  basic  nature  of  MITRE 's  C^I  FFRDC  support  to  DoD 
users  has  not  changed. •. .The  complexity  and  importance 
of  today's  C^I  systems  reaffirms  DoD's  special  needs 
and  requirements  that  are  being  performed  and 
satisfied  by  MITRE'S  C^I  FFRDC. 


At  the  first  Joint  User's  Committee  meeting  held  on 
July  19,  1990... the  continued  need  to  DoD  of  having 

MITRE'S  C^I  FFRDC  was  reviewed  in  detail. 


MITRE'S  C^I  FFRDC  is  Still  considered  to  be  a 

3 

pioneering  technical  center  in...(C  I)  systems 
engineering  technical  and  analytical  support. 

The  memorandum  made  the  following  statement  regarding 
consideration  of  alternative  sources. 

3 

Tasks  to  be  accomplished  by  MITRE 's  C  I  FFRDC  are 
assigned  only  when  the  role  is  appropriate  ^  as 
certified  by  a  senior  DoD  official  (i.e.  flag  rank  or 
SES  level  official  must  sign  the  Sole  Source 
Certification. .. .Prior  to  the  allocation  of  manpower 
resources. . .another  independent  determination  is  made 
by  the  Primary  Sponsor  and  appropriate  Sponsor  for 
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each  task  to  ensure  that  the  work  is 
appropriate. •• (i.e. ,  work  is  within  the  purpose, 
mission,  and  general  scope  of  effort  of  the  FFRDC)  and 
that  industrial  contractors  can  not  perforin  the 
required  effort. 

The  memorandum  addressed  the  efficiency  and  effectiveness  of 
MITRE  in  meeting  DoD  needs  by  stating: 

...work  performed  by  MITRE'S  C^I  FFRDC  was  determined 
to  be  very  effective,  partially  due  to  the  special 
relationship  DoD  has  with  MITRE'S  C  I  FFRDC.  This 
special  relationship  is  related  to  the  flexibility  and 
familiarity  that  MITRE'S  C^I  FFRDC  has  with  DoD  needs. 

This  makes  it  easier  for  MITRE'S  C^I  FFRDC  to  do  its 
job,  thus  increasing  the  probability  that  the  goals  of 
the  specific  project  will  be  met.  OSD  considers  this 
to  be  further  justification  for  the  continued  use  of 
MITRE'S  C^I  FFRDC,  partially  because  of  the 
government's  obligation  to  support  their  FFRDC  over 
time  at  a  level  of  activity  sufficient  to  acquire  and 
retain  technical  expertise  that  in  turn  reduces 
uncertainty  and  risk.  Industry,  in  contrast,  operates 
in  a  less  certain,  and  therefore,  higher  risk 
environment. 

The  memorandum  addressed  the  assessment  of  adequacy  of  ^  MITRE' s 
management  in  ensuring  a  cost-effective  operation  by  stating: 

Management  of  MITRE'S  C^I  FFRDC  efforts  provides 
program  guidance  to  assure  that  the  products  delivered 
adhere  to  the  services  requested.  In  addition, 
management  control  of  the  activity  has  resulted  in 
consistent  high  quality  cost  effective  product.  We 
consider  MITRE'S  management  to  be  highly  effective  and 
very  professional.  No  problems  in  this  area  have  been 
observed  by  the  Primary  Sponsor  during  the  period 
under  review.  The  quality  control  over  MITRE'S  C^I 
FFRDC  work  that  is  being  performed  by  the  MITRE  Board 
of  Trustees  and  MITRE  management  should  be  continued. 

The  memorandum  included  the  following  statement  regarding  the 
determination  that  criteria  for  establishing  the  FFRDC  were  being 
satisfied  and  that  the  sponsoring  agreement  complied  with  Office 
of  Management  and  Budget  and  FAR  requirements. 

After  detailed  examination  and  review,  the  OASD(C^I) 
has  concluded  that  all  of  the  requirements  for  using 
MITRE'S  C^I  FFRDC... are  being  met.... a  special  effort 
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was  made  to  ensure  that  all  of  the  mandatory 
requirements. . .were  incorporated  within  the  February 
11,  1991  Sponsoring  Agreement.  Compliance  with  all 
government- wide  policies  for  placing  work  at  MITRE'S 
C^I  PFRDC  is  of  utmost  importance  to  the  ASD(C^I). 


The  February  11,  1991,  Sponsoring  Agreement  contains  the 

following  statements  regarding  the  purpose  and  mission  of  the 
MITRE  C^I  Division  FFRDC. 

3 

The  primary  objective  and  mission  of  MITRE'S  Cl 
FFRDC. . .remain  unchanged.  They  are  to  provide  C^I 
general  systems  engineering,  engineering  support  and 
system  integration  support  to  the  DoD  users. ..and  to 
assist  them  in  applying  the  whole  spectrum  of  science 
and  technology  to  the  continuing  advancement  of 
military  electronic  systems.  In  performing  this 
function,  MITRE  is  a  vital  link  between  the  Government 
and  the  scientific  and  engineering  community,  with  the 
objective  of  providing  the  soundness  technical  basis 
for  the  conception,  analysis,  selection,  design  and 
evaluation  of  Information  and  Communications  Systems. 

This  role  may  include  responsibilities  in  related 
functional  areas  such  as  navigation,  detection, 
surveillance,  identification,  threat  evaluations,  and 
warning,  as  well  as  weather  and  intelligence. 


On  October  31,  1990,  the  Army  Communications-Electronics  Command 
(CECOM)  awarded  a  3 -year  sole-source  contract,  effective 
October  1,  1990,  for  work  to  be  performed  by  the  MITRE  Washington 
C^I  Division.  CECOM  used  the  sole-source  justification  and 
statement  of  work  prepared  by  the  Air  Force  Electronics  Systems 
Division  as  the  basis  for  its  sole-source  contract  to  MITRE.  The 
CECOM  competition  advocate  advised  the  contracting  officer  to  use 
the  Air  Force  sole-source  justification  because  a  new 
justification  and  statement  of  work  would  be  "redundant”  and 
would  serve  "no  useful  purpose."  The  CECOM  contracting  officer 
prepared  a  memorandum  for  file  dated  April  27,  1990,  that  stated 
in  accordance  with  FAR  5. 202 (a) (10) ,  a  synopsis  was  not  required 
to  be  published  in  the  Commerce  Business  Daily.  A  CECOM  attorney 
adviser  issued  an  undated  opinion  that  stated  that  the  Army  was 
not  a  new  sponsor  of  an  FFRDC  in  the  sense  that  it  must  comply 
with  the  requirements  of  OFPP  Letter  84-1,  as  incorporated  by 
FAR  35.017,  for  establishing  or  changing  of  an  FFRDC.  The 
attorney  adviser  stated  that: 


While  the  Army  will  be  referred  to  as  a  sponsor  of  the 
MITRE  FFRDC,  it  is  part  of  a  multiple  sponsorship 
agreement  and  as  such  only  the  primary  sponsor 
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(ASD(C^I))  is  the  "sponsor"  for  purposes  of  the  Policy 
Letter  and  Regulation. .. .Additionally,  since  this 
FFRDC  has  been  previously  established,  clearly  the 
Army  is  not  "establishing  an  FFRDC"  within  the  meaning 
of  FAR  35.017-2.  Likewise,  since  the  realignment  of 
the  MITRE  FFRDC  into  two  divisions  will  not  change  the 
basic  purpose  and  mission  of  the  FFRDC... the 
procedures  under  FAR  35.017-2  are  inapplicable. 

The  3 -year  cost-plus-fixed  fee  contract  that  CECOM  awarded  was 
estimated  to  cost  $480  million. 

Analvaia  cf  the  1990  realignment  and  contract.  We  concluded 
that  the  review  conducted  by  the  Office  of  the  Assistant 
Secretary  of  Defense  (C^I)  also  did  not  establish  that  MITRE 
support  was  the  only  or  the  most  cost-effective  source  for  the 
support  services.  The  review  did  not  clearly  differentiate 
between  work  appropriate  for  MITRE  and  work  appropriate  for  a 
non-FFRDC.  The  review  did  not  provide  sufficient  justification 
that  private-sector  firms  or  added  in-house  personnel  could  not 
perform  all  or  part  of  the  work.  Also,  the  cost-effectiveness  of 
MITRE  performance  versus  non-MITRE  sources  was  not  formally 
documented.  The  assessments  of  operational  efficiency  and 
effectiveness  were  not  based  on  objective  performance  measures. 
No  minutes  were  prepared  to  document  the  reviews  and  decisions  of 
the  Joint  Users  Committee  at  its  July  19,  1990,  meeting  as  to  the 
need  for  and  effectiveness  of  the  MITRE  C^I  FFRDC.  The  review 
did  not  discuss  the  controls  established  by  MITRE  to  ensure  cost- 
effective  operations.  Moreover,  the  review  did  not  adequately 
address  whether  the  FAR  criteria  for  establishing  the  FFRDC 
continued  to  be  met. 

Manaaemewk  comments  and  Audit  Response 

Management  commanta .  The  Deputy  Assistant  Secretary  of 
Defense  (Strategic  and  Tactical  C^)  disagreed  with  the  audit 
conclusion  that  the  comprehensive  review  conducted  by  his  office 
did  not  establish  that  MITRE  was  the  only  and  the  most  cost- 
effective  source  for  the  support  services.  He  stated  that  the 
conclusion  was  not  supported  by  facts  presented  in  the 
comprehensive  review.  He  further  stated  that  MITRE  is  not  used 
if  Government  capability  exists  or  industry  can  do  the  job  as 
effectively. 

Audit  response.  The  comprehensive  review  document  did  not 
address  the  feasibility  and  cost-effectiveness  of  bringing 
in-house  any  of  the  long-term  support  provided  by  MITRE. 
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The  Air  Force  Audit  Agency  determined  in  a  1991  audit  report 
that  non-FFRDC  contract  support  was  less  costly  than  support 
provided  by  MITRE  (See  Appendix  E  for  details) .  Also,  an 
Air  Force  Materiel  Command  study,  “A  Determination  of 
Military  and  Civilian  Personnel  Costs  as  Related  to  a  Member 
of  Technical  Staff,"  June  1992,  reported  that  comparable  in- 
house  support  was  about  $35,000  to  $55,000  .less  costly  than 
MITRE  support.  During  FY  1993,  the  Air  Force  initiated  a 
test  program,  CORAL  CONVERT,  to  convert  50  engineering  staff 
years  of  support  provided  by  the  MITRE  C^I  Division  FFRDC  to 
in-house  positions  at  the  Air  Force  Electronics  Systems 
Center. 

Management  cowwawta.  The  Deputy  Assistant  ^  Secretary  of 
Defense  (Strategic  and  Tactical  C^)  disagreed  with  the  audit 
conclusion  that  the  review  did  not  clearly  differentiate  between 
work  appropriate  for  MITRE  and  work  appropriate  for  a  non-FFRDC. 
He  stated  that  an  unsigned  sponsoring  agreement  attached  to  the 
comprehensive  review  did  include  a  well-defined  statement  on  the 
purpose  and  mission  of  the  MITRE  C^I  FFRDC  to  differentiate 
between  work  appropriate  for  assignment  to  MITRE  versus  a  non- 
FFRDC.  Also,  Electronic  Systems  Division  Regulation  80-1  and 
CECOM  Regulation  70-64,  "Utilization  of  MITRE  Support," 
October  11,  1990,  include  criteria  on  the  appropriate  use  of 
MITRE.  Further,  a  July  19,  1990,  Joint  Users  Committee  reviewed 
manpower  requests  in  detail,  and  only  50  of  140  new  manpower 
positions  requested  were  approved.  These  cuts  were  evidence  that 
a  clearly  defined  statement  of  work  for  MITRE  existed. 

Audit  response.  The  mission  statement  did  not  clearly 
differentiate  work  appropriate  for  the  MITRE  C^I  Division 
FFRDC  versus  work  appropriate  for  non-FFRDCs.  Electronic 
Systems  Division  Regulation  80-1  and  CECOM  Regulation  70-64 
contain  the  same  criteria,  which  can  be  broadly  interpreted 
to  cover  many  work  requirements.  An  August  6,  1990, 
Assistant  Secretary  of  Defense  (C^I)  memorandum  addressed 
decisions  made  during  the  July  19,  1990,  Joint  Users' 
Committee  meeting,  states  "Because  of  ceiling  growth 
constraints,  it  was  not  possible  to  accommodate  all  of  the 
requested  increases  for  FY  1991  MITRE  support."  The 
memorandum  did  not  refer  to  work  that  was  not  appropriate 
for  MITRE,  only  that  adequate'  amounts  of  MITRE  resources 
were  not  available. 

ManaaemAn^-  rsftminents .  The  Deputy  Assistant  Secretary  of 
Defense  (Strategic  and  Tactical  C^)  stated  that  OFPP  Letter  84-1, 
the  FAR,  and  facts  in  the  comprehensive  review  document  did  not 
require  a  market  survey  to  be  performed  or  the  requirements  to  be 
synopsized  in  the  Commerce  Business  Daily.  He  also  stated  that 
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an  informal  market  survey  was  done  for  each  project  and  a 
positive  determination  was  made  that  no  other  source  could 
perform  the  requirement. 

Audit  response.  We  revised  the  details  in  the  final  report 
on  alternative  sources  and  deleted  the  statement  on  the 
synopsis  of  requirements  in  the  Commerce  Business  Daily. 
The  comprehensive  review  document  must  include  an  analysis 
of  whether  the  unique  FFRDC  relationship  is  still  required 
and  whether  the  marketplace  has  changed  to  the  point  where 
competition  should  be  sought  for  all  or  part  of  the 
requirements.  The  comprehensive  review  document  did  not 
document  any  findings  of  informal  market  surveys  regarding 
the  limitations  of  private-sector  firms  to  perform  the 
proposed  tasks. 

Manaaemew*^  eQiwmantg .  The  Deputy  Assistant  Secretary  of 
Defense  (Strategic  and  Tactical  C^)  disagreed  with  the  report 
conclusion  that  the  cost-effectiveness  of  MITRE  performance 
versus  non-MITRE  sources  was  not  formally  documented.  He  stated 
that  numerous  documents,  meetings,  and  program  reviews  supported 
that  the  MITRE  services  were  cost-effective.  He  also  cited  the 
results  of  a  November  19,  1992,  Joint  Users'  Committee  as  support 
that  the  MITRE  services  were  cost-effective. 

Audit  response.  The  comprehensive  review  did  not  adequately 
document  the  cost-effectiveness  of  MITRE  support.  We 
requested  documentation  to  support  statements  concerning  the 
cost-effectiveness  of  MITRE-provided  services  versus 
alternative  capabilities  and  were  not  provided  any 
documentation  that  clearly  and  convincingly  demonstrated  the 
cost-effectiveness  of  MITRE' s  services.  The  Air  Force  Audit 
Agency  report  stated  that  the  average  FY  1990  cost  for  a 
member  of  technical  staff  at  MITRE  was  $151,000  versus  a 
comparative  cost  for  a  non-FFRDC  contractor  in  the  Boston 
area  of  $135,000.  Also,  an  on-going  Air  Force  program, 
CORAL  CONVERT,  shows  that  an  in-house  engineer  costs  about 
$25,700  less  than  a  comparable  MITRE  engineer.  The  basis 
for  the  November  19,  1992,  Joint  Users'  Committee  conclusion 
that  MITRE  services  were  cost-effective  are  addressed  in  a 
November  23,  1992,  memorandum  prepared  by  an  official  from 

the  DoD  MITRE  Program  Office  and  approved  by  the  ASD(C^I) . 
The  memorandum  stated: 

The  Joint  Users  Committee  discussion  turned  to 
the  reasonableness  of  MITRE'S  costs  of  goods  and 
services.  The  Army  pointed  out  that  the  new 
FFRDC  performance  appraisal  will  require  each 
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customer  to  assess  MITRE ^s  cost  effectiveness. 

The  ASD(C^I)  asked  whether  it  was  possible  to 
get  a  cost  per  skill  level  comparison.  Such  a 
comparison  would  clearly  show  MITRE  to  be  cost 
effective.  The  Air  Force  stated  that  a 
comparison  of  skill  levels  between  MITRE  and 
industry  is  difficult  because  "you  can't  get 
that  caliber  of  people  from  industry.”  MITRE 
stated  that  they  are  working  to  revamp  their 
cost  accounting  system  into  something  more 
conventional  which  would  enable  an  easier 
comparison  of  their  cost.  The  Joint  User 
Committee  members  concluded  that  MITRE ' s 
services  were  cost  effective. 

These  comments  were  21  months  after  the  February  7,  1991, 

OASD(C^I)  comprehensive  review  and,  in  our  opinion,  do  not 
support  conclusions  that  MITRE  services  were  cost-effective. 

Mwwaqaiiient  commewta-  The  Deputy  Assistant  Secretary  of 
Defense  (Strategic  and  Tactical  C^)  disagreed  with  the  conclusion 
in  the  draft  report  that  the  comments  regarding  operational 
efficiency  and  effectiveness  were  mostly  restatements  of  the 
FFRDC's  special  relationship  and  were  not  based  on  the  evaluation 
of  specific  taskings  accomplished  by  the  MITRE  C^I  Division 
FFRDC.  He  stated  that  the  comprehensive  review  concluded  that 
analyses  received  lender  prior  contracts  with  MITRE  have  been 
timely,  concise,  and  clear  and  that  this  was  the  consensus  of 
attendees  at  the  July  19,  1990,  Joint  Users'  Committee  meeting. 

Audit  response .  We  revised  our  conclusion  to  say  that  the 
assessments  of  efficiency  and  effectiveness  were  not  based 
on  objective  performance  measures.  Performance  measures  for 
efficiency  and  effectiveness  should  relate  outputs  produced 
to  the  resources  used  to  produce  them  and  the  extent  to 
which  results  have  been  achieved  or  objectives  met.  These 
measures  are  given  meaning  by  comparing  them  against  targets 
or  standards.  The  comprehensive  review  lacked  comparisons 
of  actual  results  versus  standards  to  support  the  assessment 
of  efficiency  and  effectiveness. 

Matiacrement  comiwanta  -  The  Deputy  Assistant  Secretary  of 
Defense  (Strategic  and  Tactical  C^)  stated  that,  although  no 
minutes  were  prepared  to  document  the  reviews  and  decisions  of 
the  Joint  Users'  Committee  at  its  July  19,  1990,  meeting  as  to 

the  need  for  and  effectiveness  of  the  MITRE  C^I  Division  FFRDC, 
the  Assistant  Secretary  of  Defense  (C^I)  signed  an  August  6, 
1990,  memorandum  that  summarized  the  decisions  made  during  the 
meeting. 
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Audit  response.  The  August  6,  1990,  Assistant  Secretary  of 
Defense  (C^I)  memorandum,  "FY  1991  Ceilings  for  the  MITRE 
Corporation,”  did  not  contain  any  comments  to  indicate  that 
the  need  for  or  effectiveness  of  MITRE  support  was  discussed 
during  the  meeting.  The  memorandum  only  discussed  resource 
allocation  and  funding  matters. 

Manaaem t  eowiment  s .  The  Deputy  Assistant  Secretary  of 
Defense  (Strategic  and  Tactical  C^)  commented  on  the  report 
statement,  "The  review  did  not  discuss  the  controls  established 
by  MITRE  to  assure  cost  effective  operations."  He  stated  this 
was  a  misleading  statement  and  cited  the  comprehensive  review 
paragraph  "Assessment  of  the  Adequacy  of  the  MITRE  C^I  FFRDC 
Management  in  Ensuring  a  Cost  Effective  Operation."  Numerous 
items  relative  to  all  FY  1990  contract  deliverables  were  assessed 
by  the  primary  sponsor  with  a  goal  of  measuring  their  value  and 
use.  The  assessment  was  briefed  to  the  Joint  Users  Committee  and 
provided  an  ample  basis  for  Joint  Users'  Committee  conclusions 
that  MITRE  services  were  cost-effective. 

Audit  response.  The  comprehensive  review  and  the  Deputy 
Assistant  Secretary's  comments  do  not  adequately  document 
the  controls  in  place  at  MITRE  to  ensure  cost-effective 
operations.  We  considered  the  comprehensive  review 
paragraph,  "Assessment  of  the  Adequacy  of  the  MITRE  C  l 
FFRDC  Management  in  Ensuring  a  Cost  Effective  Operation, "  in 
reaching  the  conclusion. 

Manaaem  an  t  comments .  The  Deputy  Assistant  Secretary  of 
Defense  (Strategic  and  Tactical  C^)  stated  that  the  proposed 
sponsoring  agreement  attached  to  the  comprehensive  review 
responded  to  all  requirements  of  OFPP  Letter  84-1  and 
FAR  35.017-4.  He  also  stated  that  the  comprehensive  review 
determined  that  the  criteria  used  to  establish  the  FFRDC 
continued  to  exist. 

Audit  response.  Attaching  a  proposed  sponsoring  agreement 
to  the  comprehensive  review  does  not  meet  the  requirements 
of  the  OFPP  policy  letter  and  the  FAR.  The  comprehensive 
review  did  not  present  a  clear  and  convincing  assessment 
that  the  criteria  for  establishing  an  FFRDC  were  met.  The 
comprehensive  review  did  not  establish  that  conventional 
contracting  methods  or  existing  in-house  capabilities  could 
not  effectively  satisfy  the  sponsor's  special  research  and 
development  requirements  (FAR  35.017-2 (a) ) ,  that  adequate 
controls  existed  to  ensure  the  cost  of  services  provided  to 
the  Government  were  reasonable  (FAR  35. 017-2 (e) ) ,  or  that 
the  basic  purpose  and  mission  of  the  FFRDC  was  stated 
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clearly  enough  to  enable  differentiation  between  work  that 
should  be  performed  by  the  FFRDC  and  work  that  should  be 
performed  by  non-FFRDCs  (FAR  35. 107-2 (f) ) . 

MITRE  is  a  specifically  created  non-Government  organization, 
private  in  form,  direction,  and  operation,  yet  public  in 
function.  MITRE' s  creation  resulted  from  recognition  that 
in-house  laboratories  could  not  attract  personnel  with  the 
skills  needed  for  very  advanced  work.  The  comprehensive 
review  did  not  show  that  the  circumstances  imposed  on  the 
United  States  in  1958  have  remained  unchanged.  The 
comprehensive  review  did  not  explain  why  DoD  could  not  ^ look 
more  to  its  in-house  organizations  and  non-FFRDC  private 
organizations  to  perform  required  tasks. 
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Background.  Lincoln  Laboratory  (the  Laboratory)  was 
established  in  1951  as  a  Government-financed,  university-operated 
research  center.  The  Laboratory  was  operated  by  MIT  and 
originally  designated  Project  LINCOLN.  Project  LINCOLN  was 
initiated  in  response  to  the  unexpected  explosion  of  the  atomic 
bomb  by  the  Soviet  Union  in  1949.  Military  officials  believed 
that  the  threat  of  nuclear  attack  against  the  continental  United 
States  by  high-speed  aircraft  in  large  numbers  required  a  drastic 
new  approach  to  air  defense.  Based  on  Work  at  several 
MIT  laboratories,  new  ideas  about  air  defense  theorized  that 
high-speed  computers  connected  to  radars  could  issue  command 
instructions  to  interceptors  and  missiles  with  a  minimum  of  human 
supervision. 

The  charter  for  Project  LINCOLN  provided  that  MIT  would  establish 
a  program  of  research  and  development  directed  toward  air 
defense.  The  prime  contract  for  Project  LINCOLN  was  awarded  by 
the  Air  Force,  but  it  was  to  serve  all  three  Military  Departments 
and  all  three  would  contribute  funds.  In  1958,  the  Defense 
Advanced  Research  Projects  Agency  became  a  sponsor,  and  about 
550  employees  (about  25  percent  of  the  Laboratory's  staff)  were 
transferred  to  the  MITRE  Corporation,  a  nonprofit  corporation 
separate  from  MIT  and  the  Laboratory.  Since  1971,  the  Federal 
Aviation  Administration  has  also  sponsored  work  at  the  Laboratory 
related  to  the  development  of  new  technology  for  air  traffic 
control . 

Since  1951,  the  Laboratory's  work  has  broadened  in  scope  and 
diversity.  The  July  26,  1951,  charter  for  Project  LINCOLN 
stated ; 


The  primary  mission  of  the  Project  will  be  Air 
Defense. .. .the  most  effective  way  of  pursuing  this 
mission  is  to  encompass  where  possible  any  problems 
pertinent  to  Air  Defense.  Continental  Air  Defense  is 
considered  to  be  a  specific  part  of  this  mission. .. .to 
conserve  manpower  and  resources ...  this  Project  may 
include  projects  now  covered  by. . .contract 
DA  36-039  SC-5450.  As  a  further  mission,  the  subject 
of  strategic  reconnaissance  and  intelligence  may  also 
be  incorporated. 

This  early  mission  led  to  the  design  and  initial  development  of 
the  Semiautomatic  Ground  Environment  Air  Defense  System  and 
expansion  into  ballistics  missile  research. 

The  mission  and  general  scope  of  the  program  as  described  in  the 
current  contract  states; 

The  mission  of  the  Laboratory  is  to  carry  out  a 
program  of  research  and  development  pertinent  to 
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national  defense  with  particular  emphasis  on  advanced 
electronics.  In  the  pursuit  of  this  mission,  the 
contractor  shall  have  the  following  objectives: 

To  exert  maximum  effort  toward  the  evolution  and 
demonstration  of  the  feasibility  of  advanced  system 
concepts  and  technology  in  selected  national  defense 
areas. 

To  conduct  specific  programs  of  research  and 

development  in  these  areas,  including  the  building  of 
necessary  components,  together  with  a  vigorous 
continuing  program  of  research  and  development  in  the 
fields  appropriate  to  its  mission. 

To  produce,  or  have  produced,  initial  models  of 

Laboratory-developed  equipment  suitable  for  field 
demonstration  and  test  by  appropriate  military 
services  or  agencies,  and  furnish  necessary 
procurement  information  and  advice  regarding  such 
equipment. 

To  provide  technical  advice  in  areas  of  its 

demonstrated  competence  to  the  military  services  and 
other  defense  and  government  agencies. 

To  annually  formulate  and  present  to  the  Joint 
Advisory  Committee  the  proposed  program  and  associated 
budgets  for  the  five  ensuing  years. 

The  contractor  programs  will  extend  from  fundamental 
investigations  in  science  through  the  development  of 
new,  advanced  technologies  to  the  integration  of  these 
technologies  into  new  or  existing  systems.  Technology 
areas  include  solid  state  electronics;  radar  optical 
sensors;  signal  processing;  surveillance;  military 
satellite  communications;  spacecraft;  analog  and 
digital  integrated  circuit  technology;  air  traffic 
control;  signal  intercept  technology;  high  energy 
laser  beam-control;  laser  devices;  optics;  antennas; 
electromagnetic  propagation;  and,  strategic  and 
tactical  systems  and  countermeasures* 

since  1951,  all  Air  Force  contracts  for  the  Laboratory  were  sole- 
source  with  MIT. 

Most  recent  review  and  iustif ication.  The  last  contract, 
awarded  September  28,  1989,  cited  10  U.S.C.  2304(c)(3), 

maintaining  an  essential  engineering,  research,  or  development 
capability  to  be  provided  by  an  FFRDC,  as  the  basis  for  the 
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sole-source  contract.  The  total  estimated  value  of  this 
5-year  contract  was  $2.9  billion.  The  Electronics  Systems 
Division  prepared  an  undated  periodic  review  document  and  a 
sole-source  justification  dated  January  25,  1989,  for  the  award 
of  this  contract.  The  justification  stated  that  the  services  to 
be  procured  from  the  Laboratory  were; 

. . . for  research  and  development  in  support  of 
Department  of  Defense  and  other  Government  agency 
programs,  with  particular  emphasis  on  advanced 
electronics.  The  requirement  includes  fundamental 
investigations  in  science,  from  the  development  of  new 
electronic  components  to  the  design,  development,  and 
field  demonstration  of  systems  embodying  the  new 
technology.  The  support  falls  into  the  following 
categories;  solid  state  electronics,  radar  and  optical 
sensors,  surveillance,  air  traffic  control,  signal 
processing,  military  satellite  communications, 
spacecraft,  electromechanical  control  systems,  lasers, 
high  energy  laser-beam  control,  optics,  antennas, 
electromagnetic  propagation,  and  construction/ 
modernization  of  research  facilities. 

The  sole-source  justification  also  stated  that  all  programs 
undertaken  by  the  Laboratory  were  reviewed  and  approved  by  the 
Joint  Air  Force/Defense  Advanced  Research  Projects  Agency/Army/ 
Navy  Advisory  Committee,  which  was  chaired  by  the  Commander,  Air 
Force  Systems  Command.  It  stated  that  the  work  to  be  performed 
required  freedom  from  bias  for  a  particular  design,  hardware,  or 
approach,  which  the  Laboratory  could  provide  due  to^  its 
dedication  to  Government  work  and  its  prohibition  from  being  a 
hardware  and  software  producer. 

The  justification  stated  that  a  market  survey  was  not  applicable 
and  a  notice  of  contract  action  was  not  published  in  the  Commerce 
Business  Daily  because  the  contract  requirements  were  within  the 
exception  described  in  FAR  6.302-3;  that  FAR  5.202(a) (10) 
authorized  the  contracting  officer  to  waive  the  synopsis 
requirement;  and  that  only  sole-source  work  may  be  assigned  to 
the  Laboratory,  consistent  with  its  FFRDC  designation.  The 
justification  further  stated  that  contract  costs  were  reasonable 
because; 


Resident  DCAA  auditors  and  a  resident  Administrative 
Contracting  Officer  provide,  on  a  full-time  basis,  the 
surveillance  required  to  monitor  a  cost-reimbursement 
(no-fee)  contract. 
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A  two-page  undated  review  document  that  referred  to  five  tabs  of 
attachments  was  prepared  after  the  January  25,  1989,  sole-source 
justitxcation  to  meet  the  requirements  of  FAR  35.017-4  for  a 
comprehensive  review.  For  the  sponsor's  special  technical 
mission  requirements,  the  review  stated: 


The  proximate  and  outyear  proposed  progr£un  is  reviewed 
by  the  Joint  Advisory  Committee  (JAC).  each 
year. .. .Minutes  of  the  most  current  review  prior  to 
award  of  [the]  contract  is  attached. 

Regarding  the  consideration  of  alternative  sources,  the  review 


The  JAC  [Joint  Advisory  Committee]  review  process 
assures  the  unique  resources  of  Lincoln  Laboratory  are 
applied  to  the  highest  priority  needs.  Work  that  can 
be  done  by  alternative  sources  are  winnowed  out  during 
this  process.... A  Justification  Review 
Document ...  stating  "there  is  no  other  known  source 
with  the  experienced  personnel,  unique  knowledge  and 
background  necessary  to  accomplish  these  (Lincoln 
Laboratory)  efforts..."  was  approved  by  the  Assistant 
Secretary  of  the  Air  Force  (Acquisition) "... (See 
Tab  2)  [Tab  2  was  the  sole-source  justification 
document . J 


effectiveness,  the  review  document 
annual  review  of  the  Laboratory's  efficiency  and 
effectiveness  was  performed.  The  review  document  stated  that 
continuii^  reviews  by  the  administrative  contracting  officer  and 
Defense  Contract  Audit  Agency  satisfied  the  requirement  for  an 
assessment  of  the  adequacy  of  the  FFRDC  management  in  ensurlnq  a 
opsj^ation,  and  that  the  Joint  Advisory  Committee 
review,  the  approval  of  the  sole-source  justification  by  the 

Force  (Acquisition) ,  and  the 
^  contract  constituted  the  required  determinations 
establishing  the  FFRDC  continued  to  be 
that  the  sponsoring  agreement  complied  with  the 


.  gyeluaticn  of  the  review  document  and  iuHttficatioti.  The 
review  and justification  documents did  not  provide  adequate 
support  ^r  the  award  of  a  5-year  sole-source  contract.  The 

August  31,  1989,  Joint  Advisory  Committee 
meeting,  which  were  attached  to  the  review  document,  provided  no 
details  on  the  specific  programs  being  worked  at  the  Laboratory. 
The  minutes  stated  that,  before  adjournment,  the  administrative 
agent  from  the  Electronic  Systems  Division  requested  the  Joint 
Advisory  Committee  to  approve  the  research  program  proposed  by 
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the  Laboratory  for  FY  1990,  and  that  the  Committee  members 
approved  the  program.  The  meeting  involve^d  no  questioning  of  the 
program  by  the  Committee  members .  We  determined  that ,  about 
3  months  earlier,  a  Joint  Advisory  Committee  Executive  Group 
chaired  by  the  Vice  Commander,  Electronics  Systems  Division,  had 
also  reviewed  and  approved  the  proposed  FY  1990  technical  and 
financial  plans  for  the  Laboratory.  The  minutes  of  the  earlier 
meeting  identified  eight  projects  but  did  not  .discuss  why  the 
Laboratory  was  uniquely  qualified  to  perform  the  research  or  what 
level  of  effort  was  planned  for  each.  The  minutes  of  this 
meeting  stated  that  the  group  was  briefed  by  the  Director  of  the 
Laboratory,  an  MIT  official,  on  an  ad  hoc  committee  formed  at  the 
request  of  the  MIT  Provost  in  response  to  faculty  concerns  with 
the  relationship  of  the  Laboratory  and  MIT.  The  Director  said 
that  the  committee  was  considering  three  alternatives:  continue 
the  relationship  as  is,  enter  a  more  distant  relationship,  or 
sever  the  relationship.  The  Director  requested  and  the 
Commander,  Air  Force  Systems  Command,  agreed  to  meet  with  the 
MIT  Committee  to  stress  the  importance  of  continuing  the 
relationship.  At  the  meeting,  the  Joint  Advisory  Committee 
Executive  Group  tasked  the  Laboratory  to  prepare  a  document  that 
described  new  technologies  that  the  Laboratory  transitioned  to 
industry  or  to  other  DoD  programs.  The  minutes  of  both  the 
August  31,  1991,  and  the  earlier  meetings  indicate  that  MIT  is 
setting  the  research  agenda  for  the  Laboratory  rather  than  DoD. 
Also, _  specific  mission  requirements  for  the  next  5  years  were  not 
identified  in  either  the  review  or  justification  document. 

We  examined  the  annual  project  review  of  the  Laboratory's 
efficiency  and  effectiveness  that  was  mentioned  in  the  review 
document  completed  in  November  1988.  The  annual  review  requested 
89  sponsors  of  work  conducted  at  the  Laboratory  during  FY  1987  to 
complete  a  three-page  questionnaire.  The  review  asked  for  an 
overall  (checkmark)  performance  rating  of  "unsatisfactory," 
"poor,"  "acceptable/expected,"  "good,"  "very  good,"  or 
"exceptional . "  The  review  also  asked  for  comments  on  the 
following  ^  areas,  as  applicable:  quality  of  performance, 
acceptability  of  cost  and  schedule  performance,  innovativeness 
and  forward-looking  nature  of  support,  quality  and  timeliness  of 
technical  reports,  timeliness  and  usefulness  of  status  and/or 
financial  management  reports,  receipt  of  required  reporting,  and 
other  comment  or  improvements.  Responders  were  requested  to 
provide  narrative  explanation  for  any  "unsatisfactory"  rating  and 
were  encouraged  to  discuss  the  evaluation  with  the  Laboratory 
staff.  The  Chief,  Electronic  Systems  Division  Lincoln  Laboratory 
Project  Office,  summarized  the  results  of  the  annual  review  in  a 
September  19,  1988,  memorandum  to  the  Electronic  Systems  Division 
contracting  officer.  The  summary  showed  that  71  of  89  sponsors 
responded  to  the  review  and  that  no  "unsatisfactory"  or  "poor" 
overall  ratings  were  given.  However,  deficiencies  were  listed. 
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Regarding  the  timeliness  of  technical  reports,  the  summary 
stated : 

High  quality  reports,  but  considerable  concern  from 
sponsors  about  getting  them  in  a  timely  manner •  This 
is  the  only  category  in  which  negative  comments  really 
outweighed  the  positive  comments.  This  was  due  to  the 
large  number  of  Sponsors  having  expressed 
dissatisfaction  with  the  timeliness  of  LL  [Lincoln 
Laboratory]  technical  reporting. 

The  summary  also  addressed  status  and  financial  reporting  by  the 
Laboratory  by  stating: 

Overall,  very  good.  Wide  variation  among  programs  with 
some  programs  not  receiving  formal  reports. 

Other  comments  made  by  sponsors  and  quoted  as  suggested 
improvements  in  the  summary  were: 

"LL  should  develop  clear  technical  milestones 

supported  by  work  breakdown  structure";  "Improve 
schedule  forecasting" ; . . . "Periodic  management/ 

technical  meetings  would  provide  a  more  complete 
understanding  of  program  status,  schedule  and 
milestones" ;  "Provide  progress/status/ financial 

reports"; .. .Need  a  better  scheme  for  financial 
projections  and  reporting". 

Evidence  does  not  show  that  the  Electronic  Systems  Division 

Lincoln  Laboratory  Project  Office  or  the  contracting  officer 

followed  up  the  concerns  before  the  contract  was  awarded. 

The  review  document  referred  to  the  sole-source  justification 
dated  January  25,  1989.  Thus,  the  review  document  was  prepared 
after  the  justification.  We  could  not  determine  how  long 
afterward.  Neither  the  sole-source  justification  nor  the  review 
adequately  documented  that  the  Laboratory  was  the  only  or  most 
cost-effective  source  for  accomplishing  the  taskings  assigned. 
The  justification  and  the  review  did  not  differentiate  between 
work  appropriate  for  the  Laboratory  and  research  that  could  be 
performed  by  other  universities  or  private  organizations.  The 

assessment  of  the  Laboratory's  efficiency  and  effectiveness  was 
not  based  on  any  objective  or  measurable  criteria.  The  review 
document  also  did  not  document  whether  MIT  had  established 
adequate  management  controls  to  ensure  a  cost-effective 
operation.  Reviews  by  the  Defense  Contract  Audit  Agency  and  the 
administering  contracting  officer  did  not  ensure  the  cost 
effectiveness  of  work  performed  by  FFRDCs.  The  reviews  provided 
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for  the  verification  of  costs  paid  versus  approved  rates  but  did 
not  include  the  performance  of  effectiveness  reviews.  Also,  the 
review  did  not  establish  that  the  FAR  criteria  for  establishing 
an  FFRDC  continued  to  be  satisfied  and  that  the  sponsoring 
agreement  complied  with  the  FAR.  We  concluded  that  the 
justification  did  not  adequately  support  the  award  of  a  5-year 
sole-source  contract  with  estimated  costs  of  $2.9  billion. 
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APPENDIX  O  -  DETAILS  ON  COMTRACT  WITH  SOFTWARE  ENGINEERING 
INSTITUTE 

Background.  The  Software  Engineering  Institute  (SEI) , 
Carnegie-Mellon  University,  Pittsburgh,  Pennsylvania,  was 
established  as  an  FFRDC  in  1984.  The  need  for  SEI  was  based  on  a 
1983  report  prepared  by  IDA,  another  FFRDC.  The  report  was  based 
on  a  study  conducted  by  3  IDA  and  13  industry  and  academia 
participants,  who  were  assisted  by  6  DoD  representatives  in  an 
advisory  role.  The  report  recommended  the  creation  of  a  new 
non-profit  FFRDC  to  improve  DoD  software  development  and  support 
by  inserting  modern  technology  into  the  life-cycle  process.  The 
report  recommended  locating  the  FFRDC  near  a  majpr  university,  so 
that  the  FFRDC  could  draw  on  professional  resources  as  necessary. 
The  report  recommended  that  the  new  FFRDC  accomplish  its^  mission 
by  seeking  out  appropriate  technology  to  adapt  and  engineer  to 
mission-critical  computer  resource  production  quality  and  scale; 
providing  funds,  talent,  and  support  to  selected  mission-critical 
computer  resource  projects  to  permit  them  to  use  the  best 
available  technology  in  developing  their  software  systems; 
establishing  a  standard  of  excellence  in  software  engineering 
practice;  and  becoming  a  source  of  top  quality  assistance  and 
support  for  the  entire  mission-critical  computer  resource 
development  and  support  community.  The  Advanced  Research 
Projects  Agency  became  the  DoD  sponsor  for  SEI  in  1989. 

The  1984  contract  was  competed  by  the  Electronics  Systems 
Division  among  seven  potential  sources:  Software  Engineering 
Development  Corporation;  Software  Engineering  Institute  at 
Dayton,  Ohio;  Texas  Engineering  Experiment  Station;  Georgia  Tech 
Research  Corporation;  Northeastern  University;  IIT  Research 
Institute;  and  SEI  of  Carnegie-Mellon  University.  SEI  was 
selected  on  the  basis  of  its  technical  ability  to  integrate  new 
software  technologies  and  capabilities  into  existing  DoD  systems. 

The  most  current  contract  was  awarded  on  December  13,  1989,  with 
an  effective  date  of  December  18,  1989.  The  contract  was  a  cost 
plus-fixed  fee  contract  awarded  on  a  sole-source  basis  for  a 
5-year  period. 

Basis  for  1989  contract.  The  Electronics  Systems  Division 
prepared  an  undated  review  document  and  a  sole-source 
justification  dated  March  22,  1989,  for  the  award  of  the 
contract.  The  sole-source  justification  stated  that  the  services 
to  be  procured  were: 

...general  software  engineering,  software  engineering 
research,  planning,  and  experimentation,  and  task 
engineering  related  to  mission-critical  defense 
systems . 
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APPENDIX  O  -  DETAILS  ON  CONTRACT  WITH  SOFTWARE  ENGINEERING 
INSTITUTE  (contM) 


The  justification  stated  that  SEI's  goal  was  to: 

...advance  the  practice  of  software  engineering  by: 

(1)  bringing  the  ablest  professional  minds  and  the 
most  effective  technology  to  bear  on  rapid  improvement 
of  the  quality  of  operational  software  in  mission- 
critical  computer  systems;  (2)  accelerating  the 
reduction  to  practice  of  modern  software  engineering 
techniques  and  methods;  (3)  promulgating  the  use  of 
modern  techniques  and  methods  throughout  the  mission- 
critical  defense  systems  community;  and  (4) 
establishing  standards  of  excellence  for  software 
engineering  practice.  In  pursuit  of  this  mission,  the 
Institute  shall  direct  its  efforts  in  the  areas  of 
software  technology  transition,  research,  planning  and 
education,  and  conduct  specific  programs  with  respect 
to  DoD  software  projects. 

The  justification  stated  that  no  known  source  had  the  experienced 
personnel,  unique  knowledge,  and  background  necessary  to 
accomplish  the  proposed  work  and,  therefore,  efforts  to  obtain 
competition  for  the  contract  action  were  unnecessary.  The 
justification  further  stated  that  a  market  survey  did  not  apply 
and  that  a  notice  of  contract  action  was  not  required  to  be 
published  in  the  Commerce  Business  Daily  because  FAR  5.202(a) (10) 
allowed  the  contracting  officer  to  waive  the  requirements. 
FAR  6.302-3  was  cited  as  the  authority  for  the  sole-source 
procurement.  The  contracting  officer  also  made  a  determination 
that  10  U.S.C.  2361,  which  prohibits  construction  and  research 
and  development  awards  to  colleges  and  universities  unless 
awarded  using  competitive  procedures,  did  not  apply  to  the 


renewal  contract  for 
University. 

SEI, 

the 

FFRDC 

at 

Carnegie-Mellon 

The  undated  periodic 

review 

stated 

that 

the 

Joint  Advisory 

Committee  chaired  by  the  Director  of  the  Advanced  Research 
Projects  Agency  and  representatives  of  the  supported  Military 
Departments  performed  a  semi-annual  review  of  the  current  and 
proposed  programs  and  that  the  projects  assigned  to  SEI  could  not 
be  accomplished  in-house  or  by  support  contractors.  The  review 
also  stated  that  the  sole-source  justification  approved  by  the 
Assistant  Secretary  of  the  Air  Force  (Acquisition)  concluded  no 
other  known  source  had  the  experienced  personnel,  unique 
knowledge,  and  background  to  accomplish  the  work.  The  review 
stated  that  an  annual  review  was  performed  to  address  SEI's 
efficiency  and  effectiveness,  and  that  reviews  by  a  Joint  Program 
Office,  the  Defense  Contract  Management  Command,  and  the  Office 
of  Naval  Research  ensured  that  SEI  was  a  cost-effective 
operation.  The  review  document  further  stated  that  the  contract 
constituted  the  sponsoring  agreement  contemplated  by 
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APPENDIX  O  -  DETAILS  ON  CONTRACT  WITH  SOFTWARE  ENGINEERING 
INSTITUTE  (contM) 

FAR  35.017-1,  and  the  Joint  Advisory  Committee  review,  the 
approval  of  the  sole-source  justification  by  the  Assistant 
Secretary  of  the  Air  Force  (Acquisition) ,  and  the  contracting 
officer's  award  of  the  contract  constituted  the  determination 
that  the  FAR  criteria  for  establishing  the  FFRDC  continued  to  be 
satisfied. 

Evaluation  of  review  anmiment  and  justification.  The 
periodic  review  and  sole-source  justification  documents  did  not 
support  the  award  of  a  sole-source  contract  valued  at 
$220.4  million  for  5  years.  Neither  document  identified  specific 
requirements  to  be  performed  by  SEI.  The  periodic  review  was 
prepared  to  satisfy  the  requirement  in  FAR  35.017-4  and  referred 
to  the  March  22,  1989,  sole-source  justification.  Thus,  the 
review  document  was  prepared  after  the  sole-source  justification. 
We  could  not  determine  how  long  afterward.  Furthermore,  we 
question  the  validity  of  the  statements  in  both  documents  that  no 
other  sources  can  perform  the  tasks.  In  addition  to  Carnegie- 
Mellon  University,  other  universities  and  organizations  have 
expertise  in  software  engineering  and  artificial  ^ intelligence 
applications.  We  also  determined  that  the  Joint  Advisory 
Committee  technical  reviews  focused  on  general  strategy,  actions 
accomplished,  and  relationships  between  the  FFRDC  and  industry 
and  DoD  organizations  but  did  not  have  good  criteria  for  defining 
which  research  tasks  should  be  performed  by  SEI.  The  mission  of 
the  FFRDC  was  not  stated  clearly  in  briefing  charts  and  minutes 
of  Committee  meetings  to  differentiate  between  work  that  should 
be  performed  by  the  FFRDC  and  work  that  should  be  performed  by 
non-FFRDC  sources. 

The  first  annual  survey  to  assess  the  performance  of  SEI  in 
meeting  sponsor  needs  was  not  completed  until  after  the 
December  13,  1989,  contract  award.  The  survey  asked  40  users  of 
SEI  to  assess  SEI's  performance  by  completing  a  questionnaire 
that  asked  for  multiple-choice  or  narrative  responses  or  both  to 
an  overall  rating  and  to  10  questions  or  statements.  Of  the 
32  users  that  responded  to  the  survey,  30  users  rated  SEI's 
overall  performance  as  exceptional,  good,  or  acceptable  and 
2  users  rated  SEI's  performance  as  unsatisfactory.  We  do  not 
believe  that  the  survey  results  can  always  be  relied  On  to 
provide  an  adequate  assessment  of  SEI's  efficiency  and 
effectiveness  in  meeting  sponsor  needs  because  the  ratings  are 
based  on  subjective  criteria. 

The  review  document  also  did  not  discuss  the  management  controls 
SEI  established  to  ensure  a  cost-effective  operation.  Also,  the 
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review  document  did  not  address  continued  compliance  with  the  FAR 
criteria  for  establishing  an  FFRDC  or  for  maintaining  a 
sponsoring  agreement. 

The  review  document  did  not  discuss  SEI'^s  accomplishments  towards 
achieving  any  of  the  goals  identified  in  the  1983  IDA  report. 
The  report  stated  that  during  the  first  1  to  2  years  of 
SEI  operation,  maximum  priority  should  be  given  to  establishing 
an  effective  strategy  for  inserting  technology  into  private 
mission-critical  computer  resource  software  contracting 
enterprises,  and  that  such  a  strategy  would  be  critical  to 
SEI  success.  The  report  further  stated  that  about  60  percent  of 
SEI  resources  should  be  devoted  to  identifying,  assessing,  and 
disseminating  software  engineering  technologies  to  meet 
DoD  requirements;  20  percent  of  resources  should  be  devoted  to 
software  engineering  support  of  the  Military  Departments  and 
Defense  Agencies;  10  percent  should  be  dedicated  to  developing 
and  conducting  training  courses  on  the  state  of  the  art  in 
software  engineering;  and  the  remaining  10  percent  should  be 
devoted  to  goal -directed  research  in  areas  of  software 
engineering  judged  to  have  high  potential  payoff.  We  do  not 
believe  that  the  contract  should  have  been  renewed  without  an 
evaluation  of  SEI's  performance  towards  achieving  those  goals. 
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Aerospace  Corporation  Assistant  Secretary  of  the  Space  and  Missile  Systems  Space  and  Missile  Systems 

Air  Force  (Acquisition)  Center,  Air  Force  Center,  Air  Force  Materiel 

Materiel  Command  Command 
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l^Contract  was  awarded  in  1988  to  provide  for  all  DoD  OI  requirements.  The  Air  Force  was  primary  sponsor. 

S/Contract  was  awarded  in  1990  to  provide  for  non-Air  Force  C^I  requirements.  Assistant  Secretary  of  Defense  (C^I)  was 
designated  primary  sponsor  of  the  MITRE  C^I  Division,  the  Air  Force  was  designated  as  sponsor  of  the  MITRE  Bedford  OI 
Group  for  Air  Force  systems,  and  the  Army  was  designated  as  sponsor  of  the  MITRE  Washington  C^I  Division. 


APPENDIX  O  -  SPMMARY  OF  M&KAGEMENT  COMMENTS  ON  THE  FINDIMG  AND 
AUDIT  RESPONSE 

This  section  summarizes  comments  by  the  Deputy  DDR&E,  the  Deputy 
Assistant  Secretary  of  Defense  (Strategic  and  Tactical  C*) ,  the 
Navy,  and  the  Air  Force  on  the  finding  and  our  responses  to  those 
comments . 

Manaaeme"»  fiomments  on  basis  for  sole-source — gontract 
renewal.  The  Deputy  DDR&E,  the  Deputy  Assistant  Secretary  of 
Defense  (Strategic  and  Tactical  C^) ,  the  Navy,  and  the  Air  Force 
stated  they  disagreed  with  the  finding.  They  stated  that 
documentation  of  the  review  process  could  be  improved  but  the 
existing  comprehensive  reviews  complied  with  OFPP  Letter  84-1  and 
FAR  35.017-4.  They  believed  that  the  reasons  for  continuing  the 
FFRDCs  were  valid  and  the  sole-source  justifications  did  not 
contain  unproven  statements. 

The  Deputy  DDR&E  said  that  report  statements  that  reviews  were 
done  late  or  not  at  all  did  not  consider  Steering  Group  meetings, 
acquisition  plans,  and  senior  official  oversight  of  FFlTOCs.  The 
Deputy  DDR&E  also  stated  that  restriction  of  competition  should 
not  be  an  issue  regarding  continuation  of  an  FFRDC  capability  and 
that  the  FAR  and  the  Competition  in  Contracting  Act  exempt  FFRDCs 
from  competition.  The  Deputy  Assistant  Secretary  disagreed  that 
the  sole-source  justifications  restricted  competition  and 
resulted  in  higher  costs.  The  Navy  stated  that  sponsoring 
program  officials,  contract  officials,  and  Navy  staff  officials, 
who  periodically  provided  verbal  briefings  to  senior  Navy 
acquisition  officials,  were  in  close  contact  throughout  the 
review  process  and  acted  responsibly  in  renewing  the 
CNA  contract.  The  Air  Force  stated  it  disagreed  that  program, 
contract,  and  senior  acquisition  officials  failed  to  properly 
exercise  their  supervisory  and  review  duties. 

Audit  response.  Sponsors  of  the  DoD  FFRDCs  are  required  to 
conduct  analyses  before  each  FFRDC  renewal  to  determine 
whether  the  unique  FFRDC  relationship  is  still  required  and, 
if  so,  to  justify  the  use  of  noncompetitive  procurement 
procedures .  The  comprehensive  review  must  document  the 
FFRDC' s  unique  research  mission,  its  past  performance  in 
providing  a  cost-effective  operation,  and  the  capability  of 
alternative  sources  to  provide  the  special  technical  needs. 
We  believe  that  the  comprehensive  review  should  provide  a 
convincing  argument  that; 

o  the  FFRDCs  have  special  skills  and  facilities  not 
otherwise  existing  in  the  Government  or  in  the  private 
sector,  or  likely  to  be  used  elsewhere  by  the  Government  or 
the  private  sector,  or  by  nature  not  appropriate  to  be 
performed  by  the  private  sector;  and 
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o  the  Government  cannot  attract  personnel  with  the 
technical  skills  and  talents  needed  to  specify  the  derived 
characteristics  and  capabilities  of  its  weapons  systems  and 
to  supervise  the  creation,  test,  installation,  and 
evaluation  of  possible  systems,  components,  or  materials. 

Changes  in  science  and  technological  capabilities  have 
occurred  since  the  inception  of  the  FFRDCs,  and  prior 
studies,  such  as  those  performed  during  the  1970s  by  the 
Defense  Science  Board,  have  documented  that  the  missions  of 
the  FFRDCs  could  be  performed  by  alternative  methods.  The 
comprehensive  reviews  must  carefully  describe  the 
requirements  to  be  satisfied  by  the  FFRDCs  during  the  term 
of  the  follow-on  contract  and  provide  rationale  why 
alternative  methods  cannot  work  as  effectively. 

The  finding  and  audit  conclusions  were  based  on  analyses  of 
all  information  made  available  to  us  during  the  audit. 
Senior  officials  may  have  been  briefed  on  the  ongoing 
comprehensive  reviews.  However,  we  were  not  provided 
documentation  of  the  briefings  and,  in  violation  of 
FAR  requirements,  9  of  10  sole-source  justifications  were 
signed  by  senior  officials  before  the  comprehensive  reviews 
were  completed.  We  revised  the  section  of  the  report 
related  to  competition  because  the  FFRDCs  are  exempted  from 
competition  by  the  Competition  in  Contracting  Act. 

Management  on  sufficiency  ef  reviews _ and 

justifications .  The  Deputy  DDR&E  stated  he  did  not  agree  with 
the  section  "Sufficiency  of  Reviews  and  Justifications"  and  the 
entire  section  should  be  rewritten  to  state  that  the  major 
weakness  with  the  reviews  was  that  they  did  not  address  internal 
controls  or  compliance  with  the  sponsoring  agreement.  The  Deputy 
DDR&E  stated  that  review  of  the  sponsoring  agreement  was  a  new 
requirement  under  FAR  35.017.  The  Deputy  DDR&E  also  recommended 
deletion  of  Appendix  G,  Suiomary  of  Reviews  and  Justifications, 
stating  that  the  appendix  indicates  a  fundamental 
misunderstanding  of  FFRDCs  on  the  part  of  the  auditors  and 
presents  a  distorted  picture  of  the  adequacy  of  the  comprehensive 
reviews  and  justifications.  The  Navy  stated  that  the 
comprehensive  reviews  were  adequate  and  provide  a  basis  for 
renewal  of  the  FFRDC  contracts.  The  Navy  also  stated  it  was 
interested  in  improving  the  review  process,  including 
documentation.  The  Air  Force  stated  that  Appendix  G  was  not 
based  on  fact. 
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Audit  response.  We  identified  weaknesses  in  the  five  areas 
of  the  comprehensive  reviews,  not  just  internal  controls  and 
sponsoring  agreements.  Before  the  1990  FAR  change,  review 
of  sponsoring  agreements  was  required  under  the  periodic 
review  requirements  of  OFPP  Letter  84—1.  We  revised 
Appendix  G  to  delete  references  to  the  synopsis  of 

requirements  in  the  Commerce  Business _ Daily.  ^  Appendix  G 

accurately  summarizes  the  deficiencies  identified  with  the 
comprehensive  reviews  and  sole-source  justifications 
prepared  for  the  10  FFRDCs  reviewed. 

ManaaemeTit  comments  on  reauirementa  for  PPRDCs.  The  Deputy 
DDR&E  recommended  the  paragraph  "Requirements  for  FFRDCs"  be 
deleted.  The  Deputy  DDR&E,  the  Deputy  Assistant  Secretary  of 
Defense  (Strategic  and  Tactical  C^) ,  and  the  Navy  stated  that 
FFRDC  sponsors  could  not  and  should  not  over-specify  research 
requirements  years  in  advance  due  to  changing  international 
policy  and  technology.  The  Deputy  Assistant  Secretary  stated 
that  future  requirements  for  the  MITRE  C^I  Division  FFRDC  are 
documented  in  yearly  sponsor  requests,  to  include  the  current  and 
4  outyears,  and  are  contained  in  Phase  I  and  Phase  II  requests. 
The  Deputy  Assistant  Secretary  did  agree  that  future  requirements 
should  be  addressed  in  a  substantive  way  in  future  comprehensive 
reviews.  The  Navy  stated  that  CNA  is  often  used  to  examine 
issues  that  cannot  be  specifically  defined  and  for  issues 
requiring  a  quick  response.  The  Navy  also  stated  that  renewal  of 
an  FFRDC  contract  is  based  on  a  continuing  research  ^  and 
development  need  of  the  sponsoring  agency  and  not  specific 
requirements . 

Audit  Response.  We  understand  that  specifying  studies  or 
research  efforts  aimed  at  specific  projects  or  tasks  years 
in  advance  may  not  be  useful  or  possible.  This  was  not  our 
intent.  Specific  research  tasks  should  be  identified  to  the 
extent  possible.  However,  we  believe  documenting  the 
special  or  unique  talents,  skills,  and  capabilities  expected 
to  be  necessary  to  accomplish  the  future  technical  needs  and 
special  research  requirements  of  the  sponsors  is  more 
important.  The  sponsors  should  also  document  how  much  of 
the  effort  involves  privileged  access  work  and  why  this  work 
cannot  be  accomplished  with  in-house  DoD  personnel.  Such 
detail  would  also  provide  a  measurable  basis  for  assessing 
the  potential  availability  of  alternative  sources  of 
capability  during  a  comprehensive  review.  The  Phase  I  and 
Phase  II  documents  referred  to  are  planning  and  budgeting 
documents  used  to  assign  annual  individual  projects/programs 
to  the  MITRE  C^I  Division  FFRDC.  The  Navy  did  not  provide 
any  information  on  the  nature  or  extent  of  work  that  could 
not  be  specifically  defined  or  that  required  a  quick 
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response  capability.  The  continuing  research  and 
development  needs  of  the  Navy  should  be  detailed 
sufficiently  to  provide  for  objective  considerations  of 
alternative  sources.  General  statements  that  requirements 
are  continuing  do  not  adequately  document  the  need  to 
continue  contracts  requiring  the  special  nature  of  the 
PFRDCs  or  the  expenditure  of  funds  .totaling  about 
$1.5  billion  annually.  We  revised  the  report  to  clarify  the 
importance  and  necessity  for  specifically  identifying  the 
nature  and  extent  of  future  special  requirements . 

Management:  comments  on  alternative  contractor  and  Government 
sources .  The  Deputy  DDR&E  stated  that  the  current  FFRDC 
contractor  is  not  required  to  be  the  only  source  for  the  FFRDC 
capability.  He  also  added  that  alternative  sources  should  be 
considered  for  individual  tasks. 

Audit  response.  We  agree  that  alternate  sources  can  be 
available.  However,  OFPP  Letter  84-1  presumes  in  favor  of 
existing  in-house  or  contractor  resources  for  satisfying 
special  research  and  development  needs.  Specifically,  the 
activities  of  FFRDCs  should  be  limited  to  those  for  which 
the  private  sector  has  no  competence  or  existing  capability. 
Where  no  private  sector  capability  exists  and  the  Government 
needs  the  services  performed,  the  services  may  be  done  in- 
house  or  through  an  FFRDC.  A  feature  of  primary  importance 
in  selecting  a  source  of  capability  is  the  effectiveness 
with  which  the  work  is  performed.  The  FFRDCs  should  have 
capabilities,  which  are  defined  during  the  comprehensive 
review  process,  that  make  the  FFRDCs  the  proper  choice  to 
perform  the  requirement. 

Management  eommantLa  on  oommerce  Business  Daily  synopsis. 
The  Deputy  DDR&E  stated  the  section  "Commerce  Business  Dally 
synopses"  should  be  deleted.  The  Deputy  DDR&E  also  stated  that 
synopses  are  published  at  the  discretion  of  the  sponsor  and 
contracting  officer  when  publication  may  benefit  the  Government. 
The  Deputy  DDR&E  stated  that  the  exemptions  granted  FFRDCs  are 
appropriate  as  long  as  FFRDCs  are  justifiable. 

Audit  response.  We  revised  the  section  of  the  report 
on  synopsis ing  requirements  in  the  Commerce  Business  Daily. 
We  agree  the  FAR  exemption  granted  from  synopsis ing 
requirements  is  appropriate  when  it  is  necessary  to  maintain 
an  essential  engineering,  research,  or  development 
capability  to  be  provided  by  an  FFRDC. 
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ManaqemwTt^--  eomments .  The  Navy  stated  that  although  not 
mentioned  in  the  comprehensive  reviews.  Government  pay  levels  lag 
considerably  behind  the  private  sector,  and  Government  officials 
are  well  aware  of  the  difficulties  the  Government  has  in 
recruiting  and  retaining  substantial  numbers  of  individuals  with 
post-masters  technical  degrees.  The  Navy  stated  that 
independence  and  objectivity  are  salient  factors  in  comparisons 
of  FFRDCs  to  private-sector  firms  and  in-house  resources.  The 
Navy  also  stated  the  special  competencies  and  capabilities  were 
the  important  factors.  In  the  case  of  CNA,  these  included  50 
years  experience  in  Naval  matters,  familiarity  with  fleet 
operations,  access  to  sensitive  data,  and  quick  response 
capability. 

Audit  response .  If  salary  levels  of  Government  personnel 
and  the  need  for  post-masters  technical  degrees  are  reasons 
for  continuing  the  FFRDC,  then  those  factors  should  be 
documented  in  the  comprehensive  review.  We  have  seen  no 
evidence  that  validates  the  assertion  that  an  FFRDC  is  more 
objective  than  an  in-house  source.  CNA  depends  on  the  Navy 
as  its  primary  source  of  funding,  and  such  dependence  will 
always  affect  objectivity  to  some  degree.  Further,  the 
relative  significance  of  the  need  for  independence  and 
objectivity  applies  equally  to  all  potential  sources.  In 
1976,  the  Defense  Science  Board  studied  the  issue  of  whether 
FFRDCs  bring  to  the  Government's  service  a  quality 
impossible  to  obtain  within  the  Government.  The  Defense 
Science  Board  concluded  that  the  study  and  analysis  FFRDCs, 
which  included  CNA,  had  no  distinct  differences  in  personnel 
characteristics  and  professional  qualifications.  The  most 
recent  comprehensive  review  for  CNA  did  not  document  such  a 
distinction  for  CNA,  or  why  the  Navy  could  not  establish  the 
capability  in-house  and  recruit  qualified  personnel  to  fill 
the  billets.  In  the  comprehensive  review  for  CNA,  the  Navy 
should  have  documented  that  50  years  experience  in  Naval 
matters,  familiarity  with  fleet  operations,  access  to 
sensitive  data,  and  quick-response  capability  are  factors 
that  affect  the  analysis  of  non-FFRDC  alternatives. 

Management  eomments  on  assessment  of  efficiency  and 
effectiveness .  The  Deputy  Assistant  Secretary  of  Defense 
(Strategic  and  Tactical  C^)  stated  that  the  primary  sponsor 
requires  each  project  officer  to  complete  a  three-page  evaluation 
form  covering  numerous  evaluation  factors,  and  the  primary 
sponsor  believes  the  criteria  for  rating  is  focused  and 
objective.  The  Navy  stated  that  obtaining  feedback  from  CNA 
customers  was  an  appropriate  and  widely  accepted  method  of 
determining  the  quality  and  utility  of  products  produced.  Also, 
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the  Navy  stated  that  the  FAR  was  silent  on  a  specific  method  for 
conducting  the  comprehensive  review,  thus  determination  of  a 
method  rests  with  the  sponsor. 

Audit  response.  The  performance  evaluations  did  address 
numerous  performance  factors.  However,  the  respondents  did 
not  use  standards  to  assign  ratings  to  the  performance 
factors  during  the  comprehensive  review.  After  the 
comprehensive  review,  standards  for  assigning  ratings  to 
each  performance  factor  were  added  to  Electronic  Systems 
Division  Regulation  80-1,  December  2,  1991.  The  use  of 

these  rating  standards  by  the  Office  of  the  Deputy  Assistant 
Secretary  will  better  ensure  the  objectivity  of  future 
assigned  ratings. 

We  agree  that  the  customer  is  the  most  appropriate 
.  individual  to  assess  the  efficiency  and  effectiveness  of  the 
FFRDC.  However,  the  assessment  conducted  by  the  Navy  did 
not  identify  basic  information  such  as  the  time  frames,  the 
number  or  type  of  projects,  or  the  costs  of  projects  that 
were  covered  by  the  customer  assessments.  Also,  the  Navy 
did  not  provide  any  basis  for  the  assignment  of  the  check¬ 
mark  ratings  that  were  made  by  the  customers.  This  was  the 
only  assessment  made  during  the  5-year  period.  Assessment 
of  the  efficiency  and  effectiveness  of  the  FFRDC  in  meeting 
sponsor  needs  should  be  an  on-going  function  conducted  on  a 
project  or  program  basis.  The  performance  measures  should 
relate  to  goals  and  should  be  verifiable  and  results 
oriented . 

Management  cowwienta  on  assessment  of  FFRDC  management 
controls .  The  Deputy  Assistant  Secretary  of  Defense  (Strategic 
and  Tactical  C^)  stated  that  subsequent  comprehensive  reviews 
would  document  controls  internal  to  the  Government  and  to  the 
MITRE  C^I  Division  for  ensuring  a  cost-effective  operation.  The 
Navy  stated  that  relying  on  the  work  of  administrative 
contracting  officers  or  of  the  Defense  Contract  Audit  Agency  in 
conducting  comprehensive  reviews  was  not  prohibited.  The  Navy 
also  stated  that  requiring  the  primary  sponsor  to  conduct  and 
document  the  reviews  and  audits  done  by  the  administrative 
contracting  officers  and  the  Defense  Contract  Audit  Agency  would 
be  redundant  and  excessive.  The  Navy  stated,  "Though  the  FAR 
requires  that  an  FFRDC  submit  to  the  sponsoring  agency  a  plan  for 
the  use  of  its  retained  earnings,  the  FAR  does  not  require  that 
management  fees  be  used  for  essential  operating  requirements." 
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Audit  response.  The  FFRDC  sponsor/  when  performing 
comprehensive  reviews,  should  rely  on  external  reviews,  such 
as  those  done  by  the  administrative  contracting  officers  and 
the  Defense  Contract  Audit  Agency,  to  the  extent  possible. 
The  Navy's  most  recent  comprehensive  review  for  CNA  did  not 
address  the  results  of  any  external  reviews.  The  results  of 
all  external  reviews  relied  on  should  be  summarized  and 
documented  in  the  comprehensive  review,  including  the 
nature,  extent,  and  timeliness  of  the  reviews,  reported 
results,  and  the  status  of  corrective  and  resolution 
actions.  Defense  Federal  Acquisition  Regulation  Supplement 
215.972,  "Modified  Weighted  Guidelines  Method  for  Nonprofit 
Organizations,"  requires  contracting  officers  to  consider 
whether  fees  are  appropriate  and  necessary  to  the  FFRDC . 
Based  on  Navy  comments,  we  deleted  the  phrase  "essential 
operating  requirements"  and  included  "appropriate  and 
necessary  to  the  FFRDC"  in  the  report.  Contracting  officers 
should  negotiate  fees  that  take  into  account  the  non-profit 
tax  status  of  the  FFRDC  and  how  the  management  fees  are 
used.  Also,  the  Senate  Subcommittee  on  Oversight  of 
Government  Management  report,  "Inadequate  Federal  Oversight 
of  Federally  Funded  Research  and  Development  Centers,"  July 
8,  1992,  recommended  that  OFPP  establish  Government -wide 
guidelines  and  criteria  on  reasonable  and  appropriate 
management  fees,  and  that  agencies  require  their  FFRDCs  to 
justify  their  management  fees  in  writing  and  reduce  any 
excessive  fees.  OFPP  agreed  with  the  recommendation. 

Management  comments  on  support  for  justifications.  The 
Deputy  Assistant  Secretary  of  Defense  (Strategic  and  Tactical  C  ) 
disagreed  with  the  report  statement,  "The  justifications  did  not 
contain  sufficient  facts  and  rationale  to  justify  the  use  of 
FAR  6.302-3  for  sole-source  awards  to  the  10  FFRDCs."  The  Deputy 
Assistant  Secretary  stated  that  competing  successive 
FFRDC  contracts  is  not  necessary  as  long  as  the  requirements  of 
the  comprehensive  review  are  met. 

audit  response.  We  agree  that  competing  successive  FFRDC 
contracts  is  not  necessary.  However,  to  use  a  sole— source 
justification  to  avoid  competing  a  contract,  contracting 
officers  must  sufficiently  document  the  rationale  for  making 
the  sole-source  award  decision. 

Newaaement  comments  on  future  awards .  The  Deputy  DDR&E 
suggested  the  paragraph  "Future  awards,"  be  deleted,  and  stated 
the  cost  in  terms  of  dollars  and  disruption  of  doing  an 
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unscheduled  comprehensive  review  before  assigning  any  future 
tasks  would  be  unjustifiable.  The  Navy  stated  that  FFRDCs  are 
exempt  from  competition  once  the  Government  establishes  a 
continuing  need  for  the  services  of  the  FFRDC.  The  Navy  stated 
that  the  Government  is  well  served  by  its  use  of  FFRDCs  and  that 
the  current  austere  budget  climate  necessitates  greater  reliance 
on  FFRDCs . 

Audit  response.  We  revised  the  report  and  related 
recommendation  because  DoD  sponsors  either  started  new 
comprehensive  reviews  in  FY  1993  or  planned  to  start  the 
reviews  in  FY  1994  for  the  10  DoD-sponsored  FFRDCs. 

Management  comments  on  effect  of  sole-source  contracts.  The 

Deputy  Assistant  Secretary  of  Defense  (Strategic  and  Tactical  C*) 
stated  that  sole-source  contracts  do  not  deny  establishment  of 
in-house  capability  provided  that  establishment  of  a  fully 
equivalent  capability  is  a  viable  option.  The  Deputy  Assistant 
Secretary  further  stated  that  FFRDCs  complement  in-house 
capability,  and  that  the  FFRDC  program  would  adapt  to  changes  in 
the  in-house  capability. 

The  Navy  stated  it  did  not  prepare  a  cost  comparison  on  field 
analysts  for  CNA  because  the  Navy  opposes  the  premise  that 
retaining  the  CNA  field  program  was  solely  a  matter  of  cost  and 
because  the  report  does  not  address  the  assumptions  used  to 
derive  the  dollar  amounts  reported. 

Audit  response.  Comprehensive  reviews  should  identify 
specific  long-term  requirements  that  FFRDCs  will  perform  and 
should  assess  the  feasibility  of  meeting  these  requirements 
with  in-house  personnel  or  with  contractor  support.  The 
comprehensive  reviews  did  not  adequately  define  the  work 
requirements  or  the  relative  merits  of  the  non-FFRDC 
alternatives. 

The  Navy  comprehensive  review  of  CNA  should  have  described 
the  requirement  for  the  field  analysts  and  should  have 
assessed  the  cost  of  in-house  performance.  We  believe  in- 
house  performance  is  a  viable  alternative  for  performing  the 
field  program  requirement.  Further,  the  Navy  comments 
indicate  the  field  program  is  a  critical  factor  in  the 
successful  accomplishment  of  CNA's  overall  study  and 
analysis  mission,  but  there  was  no  documentation  of  this 
statement  in  the  comprehensive  review.  We  further  clarified 
the  basis  for  the  cost  comparison  of  CNA  field  analysts 
versus  in-house  personnel  and  reduced  our  estimated 
reduction  of  costs  from  $7.8  million  ($3.9  million  per  year) 
to  $6.2  million  ($3.1  million  per  year). 
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Management  coimnanta  on  internal  controls.  The  Deputy 
Assistant  Secretary  of  Defense  (Strategic  and  Tactical  C^) 
disagreed  with  the  audit  determination  of  an  internal  control 
weakness.  The  Deputy  Assistant  Secretary  stated  that  all 
regulatory  requirements  were  complied  with  in  renewing  contracts 
or  sponsoring  agreements  with  the  FFRDCs.  The  Deputy  Assistant 
Secretary  also  stated  a  comprehensive  review  was  completed  before 
signing  a  sponsoring  agreement  in  January  1991  for  the  MITRE  C^I 
Division  FFRDC.  Further,  a  Joint  Users'  Committee  was  formed  to 
recommend  ceiling  allocations,  to  provide  policy  oversight  as  to 
the  appropriateness  of  work  assigned  to  the  MITRE  C^I  Division, 
to  assess  program  priorities,  and  to  assess  cost  reasonableness 
and  value  of  services  provided  by  the  MITRE  C^I  Division. 

The  Navy  stated  no  problem  existed  with  internal  controls  and 
that  program,  contract,  and  staff  officials  worked  closely 
throughout  the  contract  renewal  process.  Further,  the  Navy 
stated  that  the  Principal  Deputy  Assistant  Secretary  of  the  Navy 
(Research,  Development,  and  Acquisition)  and  the  Assistant 
Secretary  of  the  Navy  (Research,  Development,  and  Acquisition) 
were  kept  apprised,  through  verbal  briefings,  of  the  progress  of 
the  contract  renewal. 

Audit  response.  The  audit  report  and  management  comments 
state  that  documenting  comprehensive  reviews  needed 
improvement.  However,  sole-source  justifications  were 
certified  as  accurate  by  contracting  officials  and  approved 
by  senior  acquisition  officials,  and  contracts  were  awarded 
before  the  comprehensive  reviews  were  completed  to  support 
the  decisions  on  the  continued  need  for  FFRDC  relationships 
and  to  support  the  use  of  statutory  exemptions  from 
competition. 

Adequate  internal  controls  existed.  However,  the  controls 
were  not  followed  before  award  of  the  contracts.  We 
identified  and  reported  what  met  the  regulatory  criteria  for 
a  material  internal  control  weakness.  The  DoD  activities 
have  the  responsible  officials  for  determining  whether  and 
how  to  report  the  identified  weakness  in  controls. 
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APPENDIX  R  -  POTEKTIAL  BENEFITS  RESULTING  FROM  AUDIT 


Recommendation 

Reference  Description  of  Benefits 

1.  Program  Results.  Improves 

compliance  with  OFPP 
policy  by  performing 
comprehensive  reviews 
required  by  OFPP  and  the 
FAR.  Also  improves 
internal  controls. 


2. 


3. 


Program  Results.  Precludes 
award  or  extension  of 
contract  until  adequate 
justifications  are  prepared. 

Economy  and  Efficiency. 

Uses  less  costly  in-house 
personnel  instead  of  FFRDC 
contract  personnel. 


♦Consists  of  Navy  research,  development,  test, 
funding  of  $3.1  million  during  FY  1994  and  $3. 
FY  1995. 


Amount  and/or 
Type  of  Benefit 

Nonmonetary . 


Nonmonetary . 


Funds  put  to 
better  use  of 
$6.2  million.* 


and  evaluation 
1  million  during 
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APPENDIX  S  -  ACTIVITIES  VISITED  OR  CONTACTED 


Office  of  -the  Secretary  of  Defense 

Director  of  Defense  Procurement,  Washington,  DC 
Director,  Defense  Research  and  Engineering,  Washington,  DC 
Assistant  Secretary  of  Defense  (Command,  Control,  Communications, 
and  Intelligence) ,  Washington,  DC  ^ 

Assistant  Secretary  of  Defense  (Production  and  Logistics) , 
Washington,  DC 

Assistant  Secretary  of  Defense  (Programs,  Analysis,  and 
Evaluation) ,  Washington,  DC 

Director,  Acquisition  Policy  and  Program  Integration,  Under 
Secretary  of  Defense  for  Acquisition,  Washington,  DC 
Director  of  Net  Assessment,  Under  Secretary  of  Defense  for 
Policy,  Washington,  DC 

Director,  Defense  Acquisition  Regulations  Council,  Arlington,  VA 
Department  of  the  Army 

Director  of  the  Army  Staff,  Washington,  DC 
Army  Materiel  Command,  Alexandria,  VA 

Army  Communications-Electronics  Command,  Ft  Monmouth,  NJ 
Army  Concepts  Analysis  Agency,  Bethesda,  MD 
Defense  Supply  Service-Washington,  Washington,  DC 

Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Research,  Development,  and 
Acquisition) ,  Washington,  DC 
Chief  of  Naval  Operations,  Washington,  DC 
Chief  of  Naval  Research,  Arlington,  VA 

Department  of  the  Air  Force 

Deputy  Chief  of  Staff  for  Plans  and  Operations,  Washington,  DC 
Air  Force  Materiel  Command,  Wright-Patterson  Air  Force  Base,  OH 
Electronics  Systems  Center,  Hanscom  Air  Force  Base,  MA 
Space  and  Missile  Systems  Center,  Los  Angeles  Air  Force 
Base ,  CA 

Air  Force  Office  of  Scientific  Research,  Bolling  Air  Force  Base, 
Washington,  DC 

Defense  Agencies 

Advanced  Research  Projects  Agency,  Arlington,  VA 

Non~Defense  Federal  Organizations 

Office  of  Management  and  Budget,  Washington,  DC 
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Non-Government.  Activities 

Aerospace  Corporation,  Los  Angeles,  CA 
Arroyo  Center,  RAND  Corporation,  Santa  Monica,  CA 
Center  for  Naval  Analyses,  Alexandria,  VA 
Institute  for  Advanced  Technology,  Austin,  TX 
Institute  for  Defense  Analyses,  Alexandria,  VA 
Lincoln  Laboratory,  Massachusetts  Institute  of  Technology, 
Lexington,  MA 

Logistics  Management  Institute,  Bethesda,  MD 
MITRE  Corporation,  Bedford,  MA 

National  Defense  Research  Institute,  RAND  Corporation, 

Santa  Monica,  CA 

Project  AIR  FORCE,  RAND  Corporation,  Santa  Monica,  CA 
Professional  Services  Council,  Washington,  DC 
Software  Engineering  Institute,  Carnegie-Mellon  University, 
Pittsburgh ,  PA 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 

Director  of  Defense  Procurement 

Director,  Defense  Research  and  Engineering 

Comptroller  of  the  Department  of  Defense 

Assistant  Secretary  of  Defense  (Command,  Control, 

Communications  and  Intelligence) 

Assistant  Secretary  of  Defense  (Program  Analysis  and  Evaluation) 
Deputy  Under  Secretary  of  Defense  for  Logistics 

Department  of  the  Army 

Secretary  of  the  Army 

Assistant  Secretary  of  the  Army  (Research,  Development, 
and  Acquisition) 

Assistant  Secretary  of  the  Army  (Financial  Management) 

Commander,  Army  Materiel  Command 

Commander,  Communications-Electronics  Command 
Director,  Defense  Supply  Service-Washington 
Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 
Secretary  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Research,  Development, 
and  Acquisition) 

Assistant  Secretary  of  the  Navy  (Financial  Management) 

Chief  of  Naval  Research 

Auditor  General,  Naval  Audit  Service 


Department  of  the  Air  Force 
Secretary  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Acquisition) 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and 
Comptroller) 

Commander,  Air  Force  Materiel  Command 
Commander,  Electronics  Systems  Center 
Commander,  Space  and  Missile  Systems  Center 
Auditor  General,  Air  Force  Audit  Agency 


Defense  Agencies 


Director , 
Director , 
Director, 
Director , 


Advanced  Research  Projects  Agency 
Defense  Contract  Audit  Agency 
Defense  Logistics  Agency 
National  Security  Agency 
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Non-Defense  Federal  Organizations 
Office  of  Management  and  Budget 

National  Security  and  International  Affairs  Division,  Technical 
Information  Center,  General  Accounting  Office 

Chairman  and  Ranking  Minority  Member  of  Each  of  the  Following 
Congressional  Committees  and  Subcommittees: 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 

Senate  Subcommittee  on  Oversight  of  Government  Management, 
Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Operations 
House  Subcommittee  on  Legislation  and  National  Security, 
Committee  on  Government  Operations 
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Director  of  Defense  Procurement 

Deputy  Director,  Defense  Research  and  Engineering 

Deputy  Assistant  Secretary  of  Defense  (Strategic  and 
Tactical  C^) 

Office  of  the  Assistant  Secretary  of  Defense  (Production 
and  Logistics) 

Department  of  the  Army 

Department  of  the  Navy 

Department  of  the  Air  Force 

Director,  Advanced  Research  Projects  Agency 
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DIRECTOR  OF  DEFENSE  PROCUREMENT  COMMENTS 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
WASHINGTON,  DC  20301-3000 


M  2  6. 1993 


MEMORANDUM  FOR  DIRECTOR,  CONTRACT  MANAGEMENT  DIRECTORATE,  INSPECTOR 
GENERAL,  DEPARTMENT  OF  DEFENSE 

THROUGH:  CHIEF,  CONGRESSIONAL  ACTIONS  AND  INTERNAL  REPORTS 

SUBJECT:  Draft  Audit  Report  -  Sole-Source  Justifications  for 

DoD-Sponsored  Federally  Funded  Research  and  Development 
Centers  (Project  No,  lCH-5012.02) 


This  responds  to  your  request  for  comments  on  recommendation  4  of 
the  draft  audit  report.  You  recommend  that  the  Defense  Federal 
Acquisition  Regulation  Supplement  (DFARS)  be  revised  to  require  the 
renewal  of  contracts  with  DoD  Federally  Funded  Research  and 
Development  Centers  (FFRDCs)  to  be  publicized  in  advance.  You 
included  proposed  DFARS  language  in  Appendix  Q  of  the  report. 

We  do  not  agree  that  this  revision  is  appropriate,  we  are 
permitted  by  10  U.S.C,2304  (c)  (3)  to  use  other  than  coiT¥>etitive 
procedures  in  contracting  with  a  particular  FFRDC  to  establish  or 
maintain  an  essential  engineering,  research,  or  development 
capability.  Further,  41  O.S.C.  416(c)(2)  exempts  these  type 
contracts  from  the  requirement  for  publication  in  the  Commerce 
Business  Daily, 

OFPP  Policy  Letter  84-1  encourages  the  concept  of  establishing 
and  maintaining  long-term  FFRDCs  to  provide  the  continuity  necessary 
to  attract  high-qpality  FFRDC  personnel.  By  memorandum  dated 
September  16,  1992,  OFPP  reaffirmed  its  position  that  FFRDC 
contracts  are  intended  to  be  long-term,  but  emphasized  the 
iiiportance  of  conducting  adequate  market  analyses  prior  to  renewing 
FFRDCs.  Market  analysis  is  essential  to  determine  whether  the 
unique  FFRDC  relationship  is  still  needed,  or  whether  the 
marketplace  has  changed  to  the  extent  that  conpetition  should  be 
sought . 

Thank  you  for  the  opportunity  to  comment  on  the  draft  report. 

Eleanor  R.  Spector 
Director,  Defense  Procurement 


ACOUISiTION  o' 


DP  (DAR) 


Final  Report 


Deleted 


f 


135 


This  page  was  left  out  of  orignial  document 


DEPUTY  DIRECTOR,  DEFENSE  RESEARCH  AND  ENOINEERZNa  COMMENTS 


OFFICE  OF  THE  DIRECTOR  OF 
DEFENSE  RESEARCH  AND  ENGINEERING 


WASHINGTON.  OC  20901  SOSO 


0  I  C  1SS3 


MCMOKAHDim  FOR  THE  ZKSPECTOR  CEMERAL 


SUBJECT:  Rsvisv  of  ths  Draft  Audit  Report  on  Sole-Source 

Justifications  for  DoD-Sponsored  Federally  Funded 
Research  and  Developaent  Centers  (FFROCs)  (Project  No. 
1CH-5012.02) 

X  have  reviewed  the  subject  report#  and  Z  agree  with  the 
need  for  strenothened  FFRDC  aanageaent  and  better  docueentation 
to  support  decisions  relating  to  FFRDCs.  Zt  is  for  this  reason 
that  ve  issued  and  are  ispleeenting  the  DoD  FFRDC  Nanageeent 
Flan.  Z  an  concerned#  however#  with  your  interpretation  of  the 
Office  of  Federal  Procureaent  Policy  (OFF?)  letter  S4-Z  and  the 
Federal  Acquisition  Regulation  (FAR)#  and  Z  cannot  concur  with 
your  proposed  changes  to  the  FAR. 

Z  believe  that  the  DoD  is  in  cospliance  with  OFF?  Policy 
letter  S4*l  and  the  FAR#  and  that  the  FAR  is  in  compliance  with 
the  Cceipetition  in  Contracting  Act.  Z  agree#  however#  that  there 
may  be  instances  in  which  DoD  did  not  adequately  doctiment  actions 
to  support  decisions  regarding  continuation  of  FFRDCs.  This  is 
an  area  that  is  emphasised  in  the  DoD  FFRDC  Management  Plan  and 
is  receiving  increased  attention. 

Our  specific  comments  are  attached.  The  DDRiB  point  of 
contact  for  this  effort  is  Mr.  ^Robert  Ravey#  who  can  be  contacted 
at  (703)  756-0969.  1 


Attachments 


WWs 

JoM  M.  B.ctim.ky 
Dir.e«or 

D.t.nM  lt....reh  C  Bngin«.rlng 
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Final  Report 


Revised 
PP.  16  and  17 


Deleted 


Revised 

Recxxmendation  l.c. 
P.  18 

Revised 
P.  19 


Revised 
P.  20 


CDKMEKTS  OK 

THE  ZMSFECTOR  OEHERAL'8  t>Mn  AUDIT  MPORT 
OM  SOLE-SOURCE  iTUSTlFICATKWS 
FOR  DoD-SPONSOREO 

FEDERALLY  FUNDEO  RESEARCH  AND  DEVELDPKENt  CENTERS 
(PROJECT  NO.  lCK-50ia.02) 


COMMENTS  ON  K  RECOMMENDATIONS 

Pag^  IS,  Racowiandation  l.at  Partially  Concur.  In  accordonco 
with  the  DoD  FFRDC  Management  Plan,  comprehenaive  revieve 
addressing  the  criteria  contained  in  Office  of  Federal 
Procureaent  Policy  (OFPP)  Policy  Letter  $4-1  and  PAR  35.017  are 
being  accomplished  for  all  DoD  PFRDCs,  unless  a  review  was 
completed  since  January  1,  1990  (Mitre  and  CNA  come  under  this 
exception) • 


Page  15,  Recommendation  l.bs  Partially  Concur.  The  new 
comprehensive  reviews  will  address  the  guidelines  specified  in 
OFPP  Policy  Letter  S4-1  and  FAR  35.017-4.  Ve  nonconcur  with  the 
part  of  this  recommendation  that  proposes  additional  review 
requirements  not  included  in  the  Policy  tetter  or  the  FAR. 


Page  14,  Recommendation  l.c:  Nonconcur.  There  is  no  support  for 
this  recommendation  in  lav  or  regulation.  The  Competition  in 
Contracting  Act  clearly  exempts  noncompetitive  FFROC  acquisition 
from  the  requirement  for  pre-award  synopsis  in  the  Commerce 
Business  Daily  (CBD)  (Section  41  U.S.C.  414(e)(2)).  This 
statutory  exemption  is  implemented  by  FAR  5.203(a)  (10)  • 


Page  14,  Recommendation  l.d:  Concur.  The  FFRDC  Management  Plan 
requires  comprehensive  reviews  of  the  need  for  an  FFRDC  before 
its  contract  is  renewed. 


Page  14,  Recommendation  2:  Partially  Concur.  Ve  agree  with  the 
need  to  perform  comprehensive  reviews  prior  to  the  award  of  a  new 
contract.  Rowever,  to  delay  the  exercise  of  options  or 
modifications  to  ongoing  FFRDC  contracts  would  be  disruptive  to 
the  ongoing  work  of  FFRDCs  and  their  sponsors.  Such  disruptions 
could  only  be  justified  if  there  were  reason  to  believe  that  a 
review  would  not  justify  continued  sponsorship  of  the  FFRDC. 


Page  14,  Recommendation  3  s  Nonconcur.  Ve  support  the  Kavy^s 
position  that  it  would  not  be  cost  effective  for  the  Navy  to 
replace  CRA's  infrastructure  with  its  own. 
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ray*  X€i  llacoMMndatloii  4i  llonconcttr«  ••  support  ths  rssponss 
providsd  Dirsctor,  Osfsnss  Procursssnt.  "ITs  do  not  sgrss 

that  this  rsvision  is  approprists.  Ws  srs  parnlttad  by  10  O.t.C. 
3304(e)(3)  to  usa  othar  than  cospatitiva  procoduras  in 
contracting  vith  a  particular  FFRDC  to  astablish  or  saintain  an 
sssantial  snginaaring,  vMtdh,  or  davalopnant  capability. 
Further,  41  O.S.C.  414(c)(2)  axaspts  thaaa  typo  contracts  Fron 
tha  raquiranant  for  publication  in  tha  Consares  Buainass  Daily.* 


CEHERM.  COKMEMTS 

Ifa  boliava  that  va  ara  in  cosplianca  vith  OFPP  Policy  Lattar 
S4>1  and  tha  FAR  vith  raspact  to  full  and  opan  cospatition  for 
FFROCa.  Tha  applicabla  saetions  ara  guotad  balovt 

(1)  FAR  <.303<-3  statas  "Full  and  opan  eonpatitton  naad  not 
ba  provided  for  vhan  it  is  naeassary  to  avard  tha 
contract  to  a  particular  source  or  sources  in  order 
(i)...,  or  (ii)  to  establish  or  saintain  an  assantisl 
anginaaring,  research,  or  davalopnant  capability  to  ba 
provided  by  an  educational  or  other  nonprofit 
institution  or  a  federally  funded  research  and 
davalopnant  eantar.* 

(3)  OFPP  Policy  tattar  S4-1  Paragraph  4.c  statas  "Khan 
FFRDCa  ara  astabllshad,  long-tam  Govarnnant 
relationships  ara  encouraged  in  order  to  provide  tha 
continuity  that  vill  attract  high  guality  personnel  to 
tha  FFRDC.* 

(3)  FAR  35.017(4)  states  "long  tarn  relationships  batvsan 
tha  Govarnnant  and  FFROCs  ara  encouraged  in  order  to 
provide  tha  continuity  that  vill  attract  high  quality 
personnel  to  ths  FFRDC." 

FAR  35.017-4  trttlch  is  tha  basis  for  nany  of  the  objections 
raised  ^  draft  report  vas  not  in  affect  in  tine  to  affect  tha 
currant  contracts. 


SPECIFIC  COMHEMTS 

Page  il,  "Audit  Results":  Delate  all  but  first  sentence. 

Reason:  This  paragraph  nakas  three  assunptions:  (1)  tha 
coaprehenslve  raviav  oust  ba  docunantad  in  a  single  report  and 
this  report  is  tha  only  docunantation  of  tha  naad  and 
affactivanass  of  tha  PFRDC,  (2)  justifications  for  othar  than 
full  and  open  conpatition  ara  based  on  tha  currant  contractor 
being  tha  only  responsible  source,  and  (3)  the  origination  of  the 
contract  la  tha  only  tine  at  vhid:  a  sola  source  justification  is 
required.  Nona  of  these  asaunptions  is  true:  (1)  tha  progran 
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Final  Report 


Revised 


Revised 
P.  5 


docxmtntatlon  for  soot  rfRDCo  includoo  an  acquiaitlon  plan  and 
othar  daacriptiva  docuaantation  which  auat  ba  approved  by  aanior 
Banagaaant,  and  aanior  aanaganant  participataa  in  tha  ataaring 
groupa  for  all  rnU)Ca;  (2)  Juatificatlon  for  a  aola  aourca  award 
of  a  contract  to  contlnua  an  FFROC  ia  baaad  on  tha  naad  for  tha 
pnu>C  and  tha  parfonanca  of  tha  currant  rfRDC  contractor;  and 
(3)  all  taaka  aaaignad  to  an  rrRDC  muat  ba  juatifiad  aa  baing 
appropriate  for  parfomanca  by  that  FFROC. 

Page  ii«  "Suuaary  of  Racoanandationa* :  Ra%nrita  thia  paragraph  to 
•tata  tha  naad  for  Bora  thorough  docunantation  and  delate  all 
othar  coBBanta  in  thia  paragraph* 

Raaaon;  Validation  of  tha  coapatitiva  anvironaant  ia  not 
required  for  tha  contracta  that  aatabliah  PPW)C  capabilitias;  tha 
coBprahanaiva  raviava  in  queation  ware  dona;  if  tha  intent  ia  to 
reduce  tha  coat  of  aarvicaa  to  tha  govarnnant,  tha  curtai  leant  of 
contract  actiona  will  ba  countarproductiva;  raplacaBant  of  CKA 
analyata  with  in  houaa  paraonnal  ia  not  coat  affective;  and  tha 
proposed  change  to  tha  PAR  ia  not  appropriate* 


Page  7,  lat  Paragraph:  Delate  all  but  let  sentence • 

Reason:  Although  wa  agree  that  docuBsntation  to  support 
daciaiona  regarding  FFROCs  should  ba  inprovad,  tha  existing 
docunentation  cospliaa  with  tha  OFPP  Policy  Latter  §4-1  or 
PAR  35.017  depending  on  whan  tha  review  was  dona  (PM  35*017  is 
slightly  Bora  stringent  and  did  not  taka  affect  until  March 
1991).  Statesenta  regarding  reviews  "done  lata  or  not  at  all" 
neglect  tha  Steering  Group  saetings  and  docusants  such  as 
acquisition  plans  that  were  prepared  to  support  approval  of 
contract  award*  Tha  statesent  regarding  senior  officials  not 
exercising  supervisory  authority  does  not  taka  into  account  tha 
involvenent  of  these  senior  officials  in  tha  oversight  of  tha 
FPRDCs.  Tha  atatanant  regarding  validity  of  tha  reasons  used  to 
justify  tha  aola«*sourca  contracts  does  not  follow  free  what  was 
presented  before.  Restriction  of  cospatition  frop  private  sector 
fires  should  not  be  an  issue  regarding  continuation  of  an  PPRDC 
capability;  PAR  and  CICA  specifically  axaapt  PPRDCs  froB 
cospatition*  Tha  criteria  for  continuation  of  an  PPRDC  are  tha 
naad  for  tha  PPRDC  and  tha  efficiency  and  affactivaness  of  tha 
existing  PPRDC  contractor*  Tha  statesant  regarding  replacesent 
of  CKA  analysts  with  Havy  anployaas  should  ba  delated  or  included 
as  a  separata  finding. 
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Pag*  t,  "Suffleianey  of  Raviawa  and  Justification*":  Revrits 
this  paragraph  to  show  that  tha  sajor  vaaknass  in  tha  raviava  was 
that  thay  did  not  addrass  intarnal  controls  and  cospliane*  of  tha 
sponsoring  agraasant  with  FAR  35« 017-1. 

Raasoni  Tha  raviaws  war*  writtan  as  progras  avaluations«  not  as 
avaluations  of  tha  contractor's  operation.  Raviaw  of  the 
sponsoring  agreanant  is  a  new  raguiranant.  Future  raviaws. 
including  those  in  process  will  addrass  intarnal  controls  and  tha 
sponsoring  agraasant. 

Page  9.  "Raquiresants  for  FFROCs":  Delate  this  paragraph. 

Reason:  FFRDC  sponsors  cannot  and  should  not  attaspt  to 
overspecify  research,  particularly  years  in  advance  in  areas  of 
fast  changing  international  policy  or  rapidly  advancing 
technology.  OFPP  Policy  Latter  94-1  specifically  includes  quick 
response  to  user  raquiresants  as  a  characteristic  of  FFRDCs. 


Page  9,  "Altarnativa  Contractor  and  Cover nsent  sources":  Oalata 
this  paragraph. 

Reason:  Thera  is  no  raquirasant  that  the  currant  FFROC 
contractor  ba  tha  only  source  for  tha  FFRDC  capability. 
Altarnativa  sources  should  be  considered  for  Individual  tasks. 


Paga  9,  Cossarca  Business  Daily  Synopses:  Deists  all  paragra]^ 
relating  to  CBO  synopses.  Synopses  are  published  at  tha 
discretion  of  tha  Priaary  Sponsor  and  tha  contracting  officer  in 
circuBstancas  whan  the  publication  of  a  synopsis  say  benefit  the 
govamsent. 


Reason:  FAR  5. 202  (a)  (10)  exeapts  renewal  of  FFRDC  contracts  froa 
tha  raquiraaant  for  a  CBD  synopsis. 


Page  10,  Paragraph  beginning  "Mon*  of  tha  reviews...*:  Delate 
this  paragraph. 

Reason:  Tha  justification  that  only  on*  source  can  accouplish 
tha  sponsor's  individual  task  raquiresants  is  different  froa  that 
used  to  justify  entire  FFROCs.  FFRDCs  are  justified  on  the  basis 
of  aaintaining  a  capability.  The  proposed  chai^r*  to  tha  FAR  or 
Defense  FAR  Supplaaant  (DFARS)  is  not  appropriate.  Tha 
axeaptions  granted  FFRDCs  are  appropriate  as  long  as  tha  FFRDCs 
are  justifiable.  The  proposed  changes  would  underaina  the 
ability  of  the  DoD  to  keep  needed  capabilities  in  place. 
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Pa9«  13,  "Putur*  Award*"  t  l>«l«t*  thi*  paragraph. 

ftaason:  PPROC  contracts  ar«  in  accordance  with  tha  PAR,  and 
eonprahansiv*  raviaws  ara  aceoaplishad  prior  to  thair  ranawai. 
Bach  task  addad  to  an  PFRDC  contract  must  ba  justiPiad  as 
appropriata  to  partomanca  by  an  PFRDC.  Tha  cost  in  tarns  ot 
dollars  and  disruption  ot  doing  an  unschadulad  conprahansiva 
raviaw  prior  to  assigning  any  future  tasks  would  be 
unjust if iabla. 
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COMKEKTS  CH  APPENDIX  G,  SUMMARY  OP  REVIEWS  AND  JUSTIFICATIONS 
Pags  45,  Appendix  G:  Delete  this  table* 

Reason:  This  table  indicates  a  fundasental  sisundarstanding  of 
FPROCs  on  the  part  of  the  auditors  and  presents  s  distorted 
picture  of  the  adequacy  of  the  cosprehensive  reviews  and 
justifications.  Mission  requiresents  are  addressed  in  the  FFRDC 
charter  or  sponsoring  agreement.  Alternative  sources  were 
considered  either  in  the  comprehensive  reviews  or  other 
documentation  in  the  procurement  package.  There  is  no 
requirement  for  a  synopsis.  Efficiency  and  effectiveness  was 
addressed  in  all  of  the  reviews.  Compliance  with  FAR  criteria 
was  not  required  to  be  addressed  when  the  reports  were  written. 
The  only  major  deficiency  was  that  none  of  the  reviews  directly 
addressed  Internal  controls. 


COMMENTS  ON  APPENDIX  H  REGARDING  RAND  NORI 

Page  57,  paragraph  beginning  "The  Air  Force  and  the  Army...": 
Delete  the  last  sentence. 

Reason:  That  the  criteria  are  subjective  does  not  mean  that  they 
are  incorrect.  Many  of  the  studies  that  are  performed  by  FFRDCs 
are  inherently  subjective.  Designing  quantitative  criteria  would 
often  simply  quantify  subjective  arguments  without  adding  to 
their  validity. 


COMMENTS  ON  APPENDIX  I  REGARDING  IDA 

Pages  51  through  66:  Delete  the  entire  discussion  regarding  the 
CBD  synopsis  and  the  responses  to  the  synopsis. 

Reason:  The  synopsis  CBD  is  not  required,  and  the  responses  to 
the  synopses  were  handled  as  any  similar  responses  on  a  non*FFRDC 
contract  would  have  been  handled,  DSS-W  responded  properly  to 
the  contractors  that  expressed  interest  in  the  solicitation.  It 
should  also  be  noted  that  none  of  the  nine  commercial 
organisations  submitted  a  formal  protest. 


Page  66,  last  paragraph:  Delete  this  paragraph. 

Reason:  The  multiple  functions  being  performed  by  the  Director, 
DoD  FFRDC  Programs  does  nothing  but  show  the  worXload  for  this 
position.  The  Government  paid  rent  for  the  office  space  at  IDA, 
and  the  Director,  DoD  FFRDC  Programs  is  no  longer  located  at  IDA, 
The  accusation  that  the  data  call  memorandum  was  structured  to 
obtain  only  positive  comments  is  not  supported  anywhars. 
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DEPUTY  ASSISTANT  SECRETARY  OF  DEFENSE  (STRATEGIC  AND  TACTICAL  C3) 
COHMENT8 


OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 
WASHINGTON.  DC  20301  3040 

19  Kay  1993 


MEMORANDUM  FOR  THE  DIRECTOR,  CONTRACT  MANAGEMENT  DIRECTORATE, 
OFFICE  OF  TEE  DOD  INSPECTOR  GENERAL 

SUBJECT!  Draft  Audit  Report  on  Sole-Source  Justifications  for 

DoD-Sponsored  Federally  Funded  Research  and  Developnent 
Centers  (Project  number  lCB-5012.02) 

As  requested  in  your  memorandum  of  March  23,  1993,  ve  have 
reviewed  the  subject  DoD  Inspector  General  draft  audit  report. 

Our  comments  that  address  each  recommendation  are  provided  in 
TAB  A,  attached.  TAB  B  provides  general  comments  on  the  findings 
included  in  Part  II.  TAB  C  provides  specific  comments  pertaining 
to  the  1990  MITRE  C^I  PFRDC  realignment  included  in  Appendix  M. 

The  audit  report's  narrative  used  in  Appendix  M  describing 
the  February  7,  1991,  Comprehensive  Review  for  MITRE'S  C*I  FFRDC 
does  not  accurately  portray  the  scope  or  extent  of  the  review  the 
Primary  Sponsor  conducted.  As  discussed  in  TAB  C,  the  "Analysis 
of  the  1990  realignment  and  contract"  on  page  88  contains  Informa¬ 
tion  which  may  not  be  fully  supported  by  the  facts.  It  is 
suggested  that  you  re-evaluate  that  portion  of  the  report  and  make 
necessary  changes  as  appropriate  in  Appendix  M  and  the  corres¬ 
ponding  paragraphs  in  PART  ZI  of  the  report. 

In  summary,  I  believe  that  the  1990  MITRE  realignment 
strengthened  the  internal  control  procedures  relating  to  senior 
DoD  management  oversight  of  the  MITRE  C)I  FFRDC.  The  procedures 
initiated  by  the  Primary  Sponsor  and  the  Sponsors  (Air  Force  and 
Army)  after  the  1990  MITRE  realignment  are  much  more  restrictive 
than  the  management  and  oversight  requirements  included  in  the 
August  14,  1992,  DoD  FFRDC  Management  Plan. 


Deputy  Assistant  Secretary  of  Defense 
(Strategic  and  Tactical  C^) 
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DEPUTY  ASSISTANT  SECRETARY  OF  DEFENSE  (STRATEGIC  AND  TACTICAL  C3} 
COMMENTS  (cont'd) 


Final  Report 


Revised 
PP,  16  and  17 


TAB  A 


OASD(C3Z)  COMMENTS  ON  THE  RECOMMENDATIONS 
CONTAINED  WITHIN  THE  DRAFT  ADDIT  REPORT  SOLE-SOORCE 
JUSTIFICATION  Wm  DOD-SPONS(NtED  FFRDCs 


RECOMMENDATION  flAs  We  recomnend  that  DDRftE  direct  DoD  sponsors 
for  all  PFRDCs  to  prepare  new  coatprehensive  reviews* 

0ASD(C9Z)  Responsei  Concur 

The  Primary  Sponsor  of  the  NITRE  dl  FFRDC  (OASD(C3Z))  has 
already  initiated  an  effort  to  prepare  a  new  Comprehensive  Review. 
The  estimated  completion  date  is  June  IS#  1993. 

However,  it  should  be  noted  that  prior  to  signature  of  the 
MITRE  C^I  FFRDC  Sponsoring  Agreement  dated  February  11,  1991,  the 
Primary  Sponsor  completed  a  Comprehensive  Review  in  accordance 
with  the  provisions  contained  within  OFPP  Letter  84-1  and  Section 
35.017-4  of  the  Federal  Acquisition  Regulation  (FAR).  As  a  result 
of  this  Review,  a  positive  determination  was  made  on  the 
continuing  need  for  the  MITRE  C3l  FFRDC.  This  determination  was 
approved  by  the  ASD(C3I)  when  he  signed  the  Sponsoring  Agreement. 
Results  of  the  Comprehensive  Review  were  adequately  documented  in 
a  February  7,  1991  Memorandum  for  the  Record.  Exception  is  taken 
to  the  wording  in  Appendix  N  (page  86-2nd  paragraph)  that  implies 
a  staff  assistant  in  the  0ASD(C3l)  prepared  the  Comprehensive 
Review  without  any  higher  level  review  or  approval. 


RECOMMENDATION  fIBt  We  recommend  that  DDRSB  require  the  DoD 
sponsors  for  all  FFRDC  to  document  in  the  reviews  5  specified 
items  (see  page  15  and  16  of  the  draft  audit  report). 

OASD(C3l)  Responsei  Partially  Concur 

The  new  MITRE  C3i  FFRDC  Comprehensive  Review  will  address  all 
of  the  items  specified  in  OFPP  Policy  Letter  84-1  and  Section 
35.017-4  of  the  FAR.  We  nonconcur  with  this  recommendation  to  the 
extent  you  are  proposing  additional  requirements  not  currently 
included  within  the  Policy  Letter  or  the  FAR.  In  addition,  we 
take  exception  to  your  finding  (page  7-l8t  paragraph)  that  states 
"DoD  sponsor  comprehensive  reviews  did  not  meet  OFPP  Policy  Letter 
84-1  and  FAR  35.017-4  requirements”  for  any  of  the  10  DoD 
sponsored  FPRDCs.  This  is  not  an  accurate  statement  in  the  case 
of  the  MITRE  C3I  FFRDC.  Your  staff  was  provided  with  ample 
documentation  for  both  the  1988  Air  Force  and  1991  OASD(C3i) 
Comprehensive  Reviews  for  the  MITRE  C3l  FFRDC  that  met  the  full 
spirit  and  intent  of  the  FAR  and  OFPP  Policy  Letter.  Further,  a 
requirement  does  not  currently  exist  that  an  all-inclusive  single 
document  entitled  "Comprehensive  Review"  be  prepared. 
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DEPUTY  ASSISTANT  SECRETARY  OP  DEFENSE  (STRATEGIC  AND  TACTICAL  C3) 
COMMENTS  (coiit'd) 


TAB  A 


RJBCOKMKNDATION  flCi  He  reconend  that  DDRal  require  the  eppneore 
to  pubiieh  advance  notices  of  intent  in  the  Cosoaerce  Business 
Deily  and  to  perfom  aarket  surveys  to  validate  that  a  sole-source 
avard  is  warranted  because  coapetltion  is  infeasible. 

OASDfClI)  Response  t  Bonconcur 

He  believe  that  the  unique  status  of  PFRDCs  makes  this 
recommendation  totally  illoqical.  There  is  no  support  for  this 
recommendation  in  law  or  regulation.  The  Competition  in 
Contracting  Act  (CICA)  clearly  eaeapts  noncompetitive  FFROC 
acquisition  from  the  requirement  for  Commerce  Business  l>aily  (CBD) 
synopsis  (see  41  O.S.C.  416(c)(2)|#  In  addition#  Section 
5.202(a) (10)  of  the  PAR  provides  specific  regulatory  exemption  for 
this  requirement.  Section  5.205{b)  requires  placing  advance 
notices  in  the  CBO  only  when  establishing  a  new  pfroc  or  when 
changing  the  basic  purpose  and  mission  of  an  existing  PPRDC.  Mo 
such  notice  is  required  for  "other  than  commercial  or  commercial- 
type  products."  Although  the  draft  audit  report  acknowledges  the 
existence  of  these  and  other  PAR  cites#  their  intent  is  ignored. 

The  stated  objectives  of  this  audit  werei  (1)  to  determine 
whether  DoO  sponsors  were  performing  adequate  reviews  of  the  need 
for  their  FPROCs:  and  (2)  to  determine  the  adequacy  of  the  sole- 
source  renegotiation  justifications  for  the  PPRDC  contracts. 

These  objectives  are#  in  reality#  complex  policy  questions  that 
can  only  be  answered  after  addressing  a  basic  assumption  about  the 
nature  of  PFRDCs.  That  assumption  is  that  PFRDCs  are  a  wholly 
unique  type  of  national  resource  that  are  deliberately  protected 
from  normal  competitive  pressures*  This  assumption  is  not 
addressed  in  the  audit  report#  yet  this  assumption  is  well 
established  by  lav#  regulation#  and  decades  of  government 
practice.  Accordingly#  we  believe  that  this  is  a  serious  flaw 
that  diminished  the  usefulness  of  the  audit  report.  Accordingly# 
it  is  recommended  that  the  audit  report  be  revised  to  address  the 
following  three  itemss  (1)  the  unique  nature  of  the  type  of 
support  that  PPROCs  provide#  (2)  that  PFRDCs  are  intended#  by 
policy  arid  law#  to  be  organisations  with  a  unique  relationship  to 
the  government!  and  (3)  that  a  sole-source  contracting  decision  is 
an  inherently  logical  result  of  the  decision  to  maintain 
sponsorship  of  these  unique  organisations.  The  emphasis  on 
competition  inherent  in  this  recoansendation  supports  the  OASD(C)X) 
position  that  a  decision  to  continue  sponsorship  of  an  PPRDC  is 
not  severable  from  the  decision  to  sole-spurce  an  existing  PPRDC. 

The  MITRE  C’Z  PPRDC  was  created  to  support  the  DoD  in  dealing 
with  complex  C^I  technological  systems  engineering  problems  as 
they  relate  to  the  most  critical  aspects  of  national  security 
policy*  The  draft  audit  report  does  not  recognise  that  providing 
on-going  support  to  the  government  on  the  most  complex  Cll  issues 
relating  to  national  security  is  fundamentally  different  from  the 
type  of  support  provided  by  typical  service  contractors* 
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DEPUTY  ASSISTANT  SECRETARY  OF  DEFENSE  (STRATEGIC  AMD  TACTICAL  C3) 
COMMENTS  (cont'd) 


Final  Report 


Revised 

Recatinendation  l.c* 
P.  18 


TAB  A 


th«  FAR  dearly  spell  out  the  unique  relationship  that  should 
exist  between  an  FFROC  and  the  sponsoring  agency.  The  Following 
is  a  quote  Froa  Section  3S«107(a)(4)  oF  the  FARi 

*Long**ter«  relationships  between  the  Government  and  TWlDCs 
are  encouraged  in  order  to  provide  the  continuity  that  will 
attract  high-quality  personnel  to  the  PPRDC.  This 
relationship  bhould  be  oF  a  type  to  encourage  the  PFREX: 
to  maintain  currency  in  its  Field(s)  oF  expertise,  maintain 
its  objectivity  and  independence,  preserve  its  Familiarity 
with  the  needs  oF  its  sponsor(s),  and  provide  a  quick 
response  capability. * 

Section  d. 302-3  oF  the  PAR  recognises  the  unique  status  oF 
FFRDCs  and  allows  an  exception  For  other  than  Full  and  open 
competition.  Section  S. 202(a) (9)  oF  the  PAR  permits  an  award 
without  synopsising  when  the  contracting  action  is  made  under 
the  conditions  described  in  Section  d.302  oF  the  FAR.  The 
recommendation  to  require  sponsors  to  publish  sources -sought 
synopsis  and  to  perForm  market  surveys  to  validate  that 
competition  is  inFeasible.  It  also  appears  to  be  in  direct 
contradiction  to  the  PAR.  Section  6.302-3  oF  the  PAR 
(10  0.8.C.  2304(c)(3)]  statest 

•Full  and  open  competition  need  not  be  provided  For  when  it 
is  necessary  to  award  the  contract  to  a  particular  source  or 
sources  in  order  to.. .establish  or  maintain  an  essential 
engineering,  research,  or  development  capability  to  be 
provided  by  an  educational  or  other  nonproFit  institution 
or  a  Federally  Funded  research  and  development  center." 

The  lack  oF  a  requirement  to  synopsise  in  the  Competition  in 
Contracting  Act  is  indicative  oF  the  legislative  intent  when 
dealing  with  FFRDCs.  The  intent  For  the  use  oF  Section  6.302-3  oF 
the  PAR  was  included  in  Senate  Report  198-297,  which  statest 

•...  ThereFore,  the  Committee  Felt  it  was  imperative  to 
provide  agency  heads  with  a  degree  oF  Flexibility  to  establish 
or  maintain  an  essential  research  capability,  without  using 
eoaq>etitive  procedures,  to  be  provided  by  speciFic  types  oF 
institutions  or  organi sat ions •• 


RSCOMKEMDATIOM  #lDt  We  recommend  that  DORSl  discontinue  PPROCs 
that  cannot  be  adequately  justiFied. 

OASD(CSl)  Responsei  Concur  (with  word  change) 

He  concur  with  this  recommendation,  subject  to  substituting 
the  word  •Primary  Sponsor*  instead  oF  •DDRaE.*  The  Primary 
Sponsor  oF  the  FFRDC  has  this  responsibility  per  Sections  35.017-2 
and  35.017-4  oF  the  PAR. 
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DEPOTY  ASSISTANT  SECRETARY  OF  DEFENSE  (STRATEGIC  AND  TACTICAL  C3) 
COMMENTS  (cont'd) 


TAB  A 


RBCONNENDATION  |2s  We  recooimend  that  DDRftB,  ASDlPfiL),  ASN(RDA)# 
ASAF(A),  and  the  Director  DSS-Haahin^ton  not  award  any  contracts 
or  exercise  any  options  or  aodifications  to  existin9  contracts 
pending  the  completion  of  an  acceptable  comprehensive  review  and  a 
sole’^source  justification  for  the  applicable  PFRDC* 

0ASD(C3I)  Responses  Nonconcur 

We  concur  with  the  intent  of  this  recommendation  if  it  is  to 
improve  the  Comprehensive  Reviews  to  provide  a  better  basis  for 
the  sole-source  justifications*  While  the  restriction  on  awarding 
new  contracts  or  exercising  any  options  or  contract  modification 
may  be  an  effective  motivator  to  complete  these  actions*  we 
consider  the  restriction  if  applied  to  all  modifications  to 
existing  contracts  to  be  punitive*  The  fact  that  Comprehensive 
Reviews  were  not  considered  adequate  by  the  audit  report  does  not 
constitute  a  willful  action  that  requires  punitive  action* 

In  addition,  even  if  new  contracts  or  options  are  not 
exercised  until  new  Comprehensive  Reviews  are  completed, 

"contingent  planning"  for  these  contracts  should  be  carried  on. 

Not  to  do  so  would  be  placing  an  unreasonable  burden  on  the  entire 
PFRDC  community,  a  burden  which  is  not  justifiable  in  light  of  the 
fact  that  the  purpose  of  the  whole  DoD  Inspector  General’s  audit 
report  is  to  Improve  a  process  -  not  to  punish  an  entire  community 
or  deny  OoO  a  source  of  support  that  its  sponsors  strongly  endorse 
and  find  invaluable* 

The  Primary  Sponsor  of  the  NITRE  CH  FFRDC  maintains  a  firm 
position  that  the  1991  0ASD(C3i)  Comprehensive  Review  was  adequate 
and  addressed  all  of  the  mandatory  requirements  contained  within 
both  the  OFPP  Policy  Letter  84-*l  and  Section  35.017  of  the  FAR. 

The  0ASD(C3I)  also  support  the  Air  Force's  position  that  their 
Comprehensive  Review  and  sole  source  justification  for  awarding 
the  1988  contract  with  the  MITRE  Corporation  to  operate  the  C3i 
FFROC  was  adequate  and  fully  complied  with  the  requirements  of  the 
OFPP  Policy  Letter  84-1,  Section  35*817  of  the  FAR,  and  the  CICA* 

The  decision  process  for  determining  the  continued  requirement 
for  an  FFRDC  is  contained  in  the  comprehensive  review  procedure 
found  in  Section  35.017-4  of  the  FAR.  It  is  important  to  note 
that  the  purpose  of  these  reviews  is  to  review  the  use  of  and 
continuing  need  for  the  FFROC  prior  to  extending  the  contract  or 
Sponsoring  Agreement  with  an  FFROC*  The  only  alternative  to 
continued  sponsorship  is  to  transfer  sponsorship  to  another  agency 
or  to  terminate  the  FFROC.  Recompetition  of  the  contract  is  not 
provided  as  an  option*  Once  the  determination  as  to  the  use  and 
continuing  need  for  the  FFROC  has  been  made,  then  a  sole-source 
contract  negotiated  with  the  FFRDC  is  appropriate.  The  direct 
relationship  to  a  sole-source  contract  with  a  FFRDC  once 
sponsorship  has  been  determined  is  supported  by  the  unambiguous 
language  in  Sections  6*302-3(a) (2) (ii)  and  (b)(2)  of  the  FAR. 
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P.  125 


TAB  A 


RBCOHMENOATIOM  |3s  He  reconnend  that  ASM(RDA)  perfona  the 
required  cost  analyses  and  prepare  the  budget  docusents  to  replace 
the  Center  for  Naval  Analysis  field  analysts  with  less  costly 
in-house  personnel. 

0ASD(C3I)  Response:  Mo  comment 

This  recommendation  does  does  not  pertain  to  the  0ASD(C3I). 


rbcommendatzON  #4:  Me  recommend  that  the  Director  of  Defense 
Procurement  direct  the  Defense  Acquisition  Council  to  add 
provisions  to  the  Defense  PAR  Supplement  to  require  the  renewal 
of  contracts  with  DoD  PFRDCs  to  be  publicised  in  advance. 

0ASD(C31)  Response:  Nonconcur 

The  rationale  for  our  nonconcurrence  is  Included  in  the 
OASD(C3i)  response  to  Recommendation  l.C«  above.  The  0ASD(C3l) 
objects  to  setting  standards  to  DoD  different  from  those  in  the 
FAR.  Further r  we  question  whether  it  is  appropriate  for  the  audit 
report  to  establish  procurement  policy,  as  this  recommendation  is 
intending  to  do.  The  recommended  solution  to  inadequate 
management  attention  to  OFPP  or  FAR  requirements  is  to  strengthen 
management  actions,  not  to  create  additional  requirements  and 
constraints. 


REOamENDATION  ON  MATERIAL  FFRDC  INTERNAL  CONTROL  WEAKNESS: 

OASD(C3i)  Response:  Nonconcur 

With  regards  to  the  MITRE  C3I  FFRDC,  we  strongly  nonconcur 
that  there  is  a  current  weakness  in  internal  control.  The 
procedures  initiated  by  ,the  new  Primary  Sponsor  of  the  MITRE  C3i 
FFRDC  after  the  1990  MITRE  realignment  are  much  more  restrictive 
than  the  management  and  oversight  requirements  included  in  the 
August  14,  1992  DoD  FFRDC  Management  Plan.  As  the  Primary  Sponsor 
of  NITRE'S  C3l  FFRDC,  the  OASD(C3i)  reviews  every  project  to  ensure 
that  the  requirements  of  our  Sponsoring  Agreement,  ONB's  OPPP 
Policy  Letter  84-1,  and  Section  35.017-4  of  the  PAR  are  complied 
with.  A  written  Comprehensive  Review  was  prepared  prior  to 
approval  of  the  February  11,  1991  NITRE  Sponsoring  Agreement  that 
was  fully  responsive  to  all  of  the  requirements  contained  within 
the  OPPP  Policy  Letter  84-1  and  Section  35.017-4  of  the  FAR.  In 
addition,  a  Joint  Users  Committee  was  formed  that  has  the 
following  responsibilities:  (X)  reccHomend  ceiling  allocations  to 
the  Primary  Sponsor;  (2)  provide  policy  oversight  to  ensure  that 
all  NITRE  C3l  FFRDC  work  is  appropriate;  (3)  provide  an  assessment 
on  the  MITRE  C3l  FFRDC  program  priorities;  and  (4)  assess  the  cost 
reasonableness  and  value  of  the  C3l  systems  engineering  services 
provided  by  the  MITRE  C3l  FFRDC. 
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DEPUTY  ASSISTANT  SECRETARY  OF  DEFENSE  (STRATEGIC  AND  TACTICAL  C3) 
COMMENTS  (cont'd) 


TAB  8 


OASD(C)I)  GENERAL  COMMENTS  ON  THE  FINDINGS 
CONTAINED  NITHIN  THE  DRAFT  AUDIT  REPORT  ON  SOLE-SOORCE 
OOSTIPICATICM  rOR  DOD-SPONSORED  PPROCs 


ISSUES  Findings*  Page  7  of  the  16  report.  Sponsors  did  not 
adequately  document  the  basis  for  renewing  the  10  PPRDC  Sole 
Source  contracts. 

Responses  This  is  a  criticism  of  the  documentation;  it  is  not  a 
statement  that  there  was  no  basis  for  renewing  the  sole  source 
contracts.  OASD(C3I)  acknowledges  that  the  AF  In  1986  could  have 
documented  the  basis  for  renewal  of  the  contract  in  more  depth. 
However F  even  without  more  extensive  documentationr  the  sponsor 
did  establish  the  basis  for  maintaining  the  continuing  need  for 
the  MITRE  C3I  FFRDC.  As  elaborated  below,  OASD(C3I)  Is  in  the 
process  of  generating  an  updated,  more  thoroughly  documented 
comprehensive  review  (even  though  one  was  generated  2  years  ago) 
The  Air  Force  and  Army  sole  source  contract  justifications  will 
derive  substantially  from  this  effort.  Therefore,  OASD(C3I)  is 
positively  responding  to  the  indicated  need  for  more  thorough 
careful  documentation. 

ISSUE:  Findings.  Page  7  of  16  report.  Comprehensive  reviews  did 
not  meet  OFPP  Policy  Letter  84-1  and  FAR  3S.  017-4. 

Response:  Hoth  OFPP  84-1  and  FAR  35.017-4  list  the  five  areas  of 
review  that  should  be  included  in  the  Government's  comprehensive 
review  of  their  FFRDC.  In  summary  paraphrasing,  these  five  review 
requirements  ares 

1.  Special  technical  needs  and  mission  requirements 
performed  by  FFRDC  to  determine  if  and  at  what  level  they 
continue  to  exist. 

2.  Consideration  of  alternative  sources  to  meet  sponsor 
needs • 

3.  Assessment  of  efficiency  and  effectiveness  in  meeting 
sponsor  needs • 

4.  Assessment  of  adequacy  of  the  FFRDC  management  in 
ensuring  a  cost  effective  operation. 

5.  Criteria  for  establishing  FFRDC  continues  to  be 
satisfied  and  Sponsoring  Agreement  meets  35.017-1. 

On  7  February  1991  OASD(C31)  in  a  memorandum  entitled,  "Review  of 
DOD'S  Continued  Need  for  MITRE'S  C3I  FFRDC."  OASD  (C3I)  (TAB  A) 
reported  on  the  results  of  the  comprehensive  review  of  the  MITRE 
C3I  FFRDC  conducted  by  OASD(C3I)  prior  to  the  signing  of  the 
DoD/MlTRE  sponsoring  agreement  dated  11  February  1991.  The  1991 
comprehensive  review  addressed  each  of  the  5  areas  of  review  cited 
above  from  OFPP  letter  84-1  and  FAR  35.017-4.  In  addition  to  the 
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ASD(C3X)f  a  copy  of  that  review  was  distributed  to  DASD(PCrL)#  Air 
Porce/£SCr  Army/CECOH,  DDME,  and  the  DoD  IG. 

All  of  the  requirements  identified  for  the  continued  need  and  use 
of  the  MITRE  C3I  PFRDC  were  met  and  it  was  so  stated  in  the 
comprehensive  review.  It  was  also  pointed  out  that  since  the 
Primary  Sponsor  of  the  MITRE  C3I  PFRDC  does  not  and  will  not  have 
.a  direct  contract  with  MITREr  special  efforts  were  made  to  assure 
that  the  Sponsoring  Agreement  between  MITRE  and  ASD(C3I),  dated  11 
February  1991  (TAB  B),  embodied  all  of  the  mandatory  requirements 
relative  to  the  need  and  use  of  the  PFRDC  contained  in  OPPP  84-*! 
and  the  FAR. 

Since  the  issuance  of  the  DoD  IG  Audit  Report  draft#  OASD(C3X)#  in 
a  spirit  of  cooperation  with  DDRCiB  and  the  DoD  IG#  has  agreed  to 
perform  a  new  comprehensive  review  of  MITRE  even  though  the  91 
review  satisfied  84-1  and  FAR  requirements.  The  new  review  is 
ongoing  and  is  expected  to  be  completed  and  approved  in  the 
spring  of  93.  It  addresses  each  of  the  criteria  identified  above 
and  will  document  the  satisfaction  of  the  mandatory  requirements 
identified  in  84-1  and  the  FAR. 

ISSUES  Findings.  Page  7  Audit  Report.  Sole  source 
justifications  contained  unproven  statements.  Reasons  used  to 
justify  sole  source  for  the  10  FFRDCs  were  not  valid.  Therefore# 
competition  was  restricted  from  private  sector  and  probably 
resulted  in  higher  costs. 

Responsei  OASD(C3I)  believes  that  the  reasons  used  to  justify 
sole  source  for  the  10  FFRDCs  were  valid.  The  fact  that  they  were 
not  well  documented  does  not  mean  that  the  rationale  was  *'not 
valid.* 

0AS0(C3X)  does  not  concur  with  the  statement  that  c<»kMtition  was 
restricted  from  private  sector  and  probably  resulted  in  higher 
costs.  The  IG  did  not  do  an  independent  review  of  costs;  it 
appears  that  they  may  have  accepted  1  or  2  data  points  provided  by 
external  sources.  Professional  analysis  would  require  dealing 
with  job  comparability  and  value  received  as  well  as  dollar  costa. 

ISSUES  Page  9  of  Audit  Report.  Did  not  detail  future 
requirements. 

Responses  The  future  requirements  for  the  MITRE  FFRDC  are 
documented  yearly  in  the  sponsors*  requests  for  MITRE  support  and 
are  contained  in  what  is  referred  to  as  the  Phase  I  and  Phase  II 
requests  for  MITRE  support  in  both  the  ESC  Regulation  80-1  and 
CBCOH  Regulation  70-64. 

For  each  project  requested  by  the  sponsor#  his  Phase  I/II  request 
states  the  requirements  not  only  for  the  current  year  but  for  the 
following  4  years.  This  has  been  a  long  standing  requirement  for 
the  Government  users  of  MITRE  and  continues  to  bs  a  regulatory 
CECQM  Regulation  70-64  and  BSC  Regulation  80-1)  requirement. 
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Furthermore,  the  existing  and  growing  need  Cor  C3I  as  a  "force 
multiplier  when  faced  with  force  reductions^  for  current  systems 
modernitation#  dual  use  technology  applications,  and  broad  based 
interoperability  integration  efforts  in  support  of  joint 
operations  make  the  future  needs  for  the  MITRE  PFRDC  more 
compelling  that  ever. 

The  0ASD(C3I)  agrees  that  future  requirements  should  be  addressed 
in  a  substantive  way.  The  updated  comprehensive  review  currently 
being  prepared  will  do  so  and  will  include  a  discussion  and 
projection  of  future  needs.  It  must  be  realised  that  detailed 
needs  cannot  be  predicted  S  years  hence  with  precision 
particularly  in  these  rapidly  changing  times.  Basic  needs  are 
derived  from  increased  emphasis  and  reliance  upon  C3I  and  on  the 
key  role  the  MITRE  C3I  FFRDC  plays  in  the  C3I  arena. 

ISSOB}  Page  II  and  Appendix  M,  page  SS,  Audit  Report.  Although  pp^  ^  ^3 

the  periodic  review  assessed  operational  efficiency  and 
effectiveness,  the  review  was  not  based  on  objective  criteria. 

Responses  The  primary  sponsor  requires  that  each  government 
project  officer  annually  completes  a  three  page  evaluation  form 
that  ranks  MITRE  on  numerous  factors,  such  as:  (1)  the  quality, 
value  and  timeliness  of  deliverables;  (2)  an  assessment  of  MITRE'S 
efficiency  and  effectiveness  in  meeting  DOD  needs;  and  (3)  an 
assessment  of  MITRE  management  In  ensuring  a  cost  effective 
operation. 

These  assessments  encompass  the  MITRE  C3I  FFRDC  support  of  program 
objectives,  working  relationships,  project  productivity,  quality 
of  projects,  project  staffing,  and  a  summary  appraisal.  Using 
this  approach  (overall  performance  based  on  the  total  number  of 
questions  answered)  resulted  in  98.9%  of  questions  rated  as 
"Exceptional/Always"  or  "Acceptable/Usually".  Only  1.1%  were 
rated  as  "Unsatisfactory/Rarely."  Critical  remarks  by  the 
sponsors  have  been  taken  seriously  by  the  Army,  the  Air  Force,  the 
Primary  Sponsor  and  MITRE  and  have  resulted  in  remedial  action  in 
all  cases  where  appropriate,  and  sponsor/MITRE  dialogue  to  achieve 
a  common  understanding  in  those  cases  where  remedial  action  was 
not  appropriate  (e.g.,  miscommunication). 

OASO{C31)  believes  the  criteria  for  rating  to  be  both  focused  and 
objective.  The  current  ongoing  craprehensive  review  contains 
appended  material  that  describes  the  actions  taken  by  the  NITRE 
(C32)  FFRDC  as  reported  to  0ASD(C3I),  in  response  to  the  1.1% 
unsatisfactory  ratings  for  PY  91.  Evaluations  for  FY  92  are 
currently  ongoing  and  will  be  completed  in  early  summer  93.  As 
importantly,  the  FFRDC  relationship  assures  that  there  is 
continuous  dialogue  between  sponsors  and  NITRE'S  C3I  FFRDC 
concerning  performance  and  that  any  issues  are  addressed  without 
waiting  for  formal  report  card  submissions. 
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ISSOEi  Pages  11,  12,  and  BS  of  Audit  Report.  The  comprehensive 
revieir  did  not  address  controls  established  by  the  FFRDC  to  ensure 
a  cost-effective  operations 

Responsei  This  is  also  a  criticism  of  process  and  documentation 
and  does  not  establish  that  adequate  controls  are  not  in  place  in 
the  Office  of  the  Primary  Sponsor p  in  the  Air  Porcep  in  the  Army# 
or  at  MITRE. 

As  a  general  comment,  both  the  primary  sponsor  and  the  respective 
Air  Force  or  Army  sponsor  monitor  the  C3I  FFRDC  program  execution 
during  the  year  to  assure  that  the  DoD  is  receiving  acceptable 
products  at  reasonable  costs*  MITRE  submits  detailed  financial 
and  manpower  reports  to  every  project  on  a  routine  monthly  basis 
and  costs  are  reviewed  by  both  the  appropriate  sponsor  and  the 
DCAA  prior  to  payment  to  the  MITRE  C3I  FFRDC. 

The  currently  ongoing  comprehensive  review  of  MITRE* s  C3I  FFRDC 
will  document  the  controls  both  internal  to  the  Government  and  to 
MITRE  for  assuring  cost  effective  operation.  A  document  from 
MITRE  describing  the  controls  in  place  to  assure  accurate 
accounting  records;  sound  procurement  practices;  reasonable 
overhead  costs;  that  management  fees  are  for  essential  operating 
requirements;  and  other  related  issues,  will  be  appended  to  the 
comprehensive  review. 

ISSUES  Page  13  of  Audit  Report.  DoD  sponsors  should  compete  to 
the  maximum  extent  possible  mission  requirements  or  technical 
needs  requiring  FFRDC  performance. 

Justifications  did  not  contain  sufficient  facts  and  rationale  to 
justify  the  use  of  FAR  6.302.3  for  sole  source  awards  to  the  10 
FFRDCS. 

Response;  After  an  FFRDC  has  been  competed  and  established,  the 
FAR  is  very  clear  that  it  is  not  necessary  to  compete  successive 
contracts  In  order  to  maintain  the  FFRDC  as  long  as  there  is  no 
change  in  the  basic  purpose,  mission,  and  scope  of  that  FFRDC  and 
as  long  as  the  requirements  of  the  comprehensive  review  are  met 
fe.g.,  assurance  of  cost  effective  opcrstions).  That  is  precisely 
the  latter  and  intent  of  FAR  6.302. 3. 

OASD(C3I)  agrees  that  DOD  sponsors  should  compete  to  the  maximum 
extent  possible  if  it  is  determined  by  DoD  that  a  new  FFRDC  needs 
to  be  established  to  meet  mission  requirement  or  technical  needs. 
But  the  FAR  specifies  that  competition  is  not  required  in  cases  of 
existing  FFRDCs  whose  mission  and  scope  remain  unchanged. 

Furthermore,  with  respect  to  maximum  competition,  the  FAR 
Competition  in  Contracting  Act,  and  DoO  policy  is  to  recognize 
that  special  situations  do  exist  where  the  Government  is  better 
served  and  it  is  in  the  public  interest  if  formal  competition  is 
not  required.  In  the  case  of  FFRDCs,  a  long-term,  stable 
relationship  is  a  fundamental  policy  objective.  FFRDCs  have  long 
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been  uniformly  recognized  as  national  assets  providing  a  national 
capability  and  recognized  as  being  of  high  value#  providing  high 
quality  work#  with  positive  evidence  of  the  need  to  retain  them 
even  during  periods  of  austerity.  Regarding  this  evidence,  it 
should  be  noted  that  the  MITRE  C3I  PFRDC  work  is  industrially 
funded  and#  that  program  managers  throughout  the  DoD  continue  to 
choose  to  use  their  increasingly  scarcer  funds  to  obtain  MITRE 
support.  The  result — continuing  strong  demand. 

ISSUE:  Page  14  of  Audit  Report.  DOD  4502*2  CAAS  states 
Government  should  establish  in-house  capability  ASAP  unless  it 
muld  not  be  coat  effective. 

Response:  Sole  source  contracts  do  not  deny  establishment  of  in- 
house  capability;  the  real  issue  is  whether  or  not  establishment 
of  a  fully-equivalent  capability  is  a  viable  option;  PFRDCs  work 
on  the  government  side  of  the  table  complementing  the  in-house 
capability.  If  that  capability  were  to  change/increase#  the  PFRDC 
program  would  adapt* 
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0ASD(C3I)  SPBCiriC  COMHEMTS  ON  APPENDIX  N 
CONTAINED  NITBIN  OF  THE  INtAFT  AODIT  REFCNtT  ON  SOLS-SOORCE 
JUSTIFICATION  KM  DOO-SFONSORED  FFROCB 


ISSUE  ON  PAGE  88t  ANALYSIS  OF  TBS  1990  REALIGNMENT 

The  first  sentence  reads t  "He  concluded  that  the  review 
conducted  by  the  OASD(C3i)  also  did  not  adequately  establish 
that  MITRE  support  was  the  only  or  the  most  cost-effective 
source  for  the  support  services." 

OASD(C3i)  Responses  This  statement  is  inaccurate*  It  is  not 
supported  by  the  facts  included  in  the  Comprehensive  review* 

As  stated  on  page  4  of  the  1991  Comprehensive  Review^  a  Sole 
Source  Certification  Form  for  each  project  must  be  signed  by  a 
senior  DoD  official  (i.e*#  flag  rank  or  S£S  level)  before  it  can 
be  placed  on  contract*  The  following  statements  are  included 
within  this  signed  certification:  "Only  MITRE  can  satisfy  the 
government's  recpiirement  for  this  particular  effort"  and  "There 
are  no  other  known  contractors  with  the  experienced  personnel# 
technical  objectivity,  and  necessary  capabilities  to  perform  this 
effort." 

In  addition,  Page  4  of  the  Comprehensive  review  describes  an 
independent  assessment  performed  by  the  Primary  Sponsor  to  ensure 
that  all  work  is  appropriate  (within  the  FFRDC's  purpose,  mission 
and  general  scope  of  effort,  cost  effectiveness,  etc*)*  Six 
specific  factors  were  listed,  all  of  which  recjuired  a  favorable 
determination.  As  stated  on  page  it 

"If  a  favorable  determination  is  not  made,  then  DoD  policy 
is  to  do  the  work  either  in-house  or  initiate  efforts  to 
award  a  competitive  contract  with  for-profit  contractors* 

In  addition,  MITRE* s  C^I  FFRDC  is  not  used  if  a  government 
capability  exists  or  if  industry  can  do  the  job  as  effectively 
and  without  conflict  of  interest*" 

The  above  plus  other  safeguards  described  in  the  Comprehensive 
Review  and  the  documents  it  references  more  than  adequately 
establishes  the  fact  that  NITRE  was  the  only  effective  source  to 
accomplish  the  work*  In  fact,  the  OASD(C3l)  challenges  the  DoD  IG 
to  find  another  FFRDC  sponsor  (government-wide)  that  does  more  to 
ensure  that  only  appropriate  work  is  assigned  to  their  FFRDC. 


ISSOB  ON  PAGE  88l  AMAI.YSIS  OF  THE  1990  REALIGNMENT 

The  second  sentence  reads:  "The  review  did  not  clearly 
differentiate  between  work  appropriate  for  NITRE  and  work 
appropriate  for  a  non-FFRDC*" 

0ASD(C3I)  Responses  This  statement  is  not  supported  by  the 
facts  included  in  the  Comprehensive  Review* 
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As  Stated  on  page  1  of  the  Comprehensive  reviewr  the  DoD 
Sponsoring  Agreement  for  the  HITRE  FFHOC  was  attached. 
Accordingly#  the  Sponsoring  Agreement  (even  though  not  yet  signed) 
was  an  integral  part  of  the  Comprehensive  Review.  The  Sponsoring 
Agreement  includes  a  well  defined  statement  on  the  purpose  and 
mission  of  the  MITRS  C^I  FFROC  that  more  than  adequately  allows 
program  managers  and  senior  management  reviewers  to  differentiate 
between  work  appropriate  for  MITRE  and  work  appropriate  for  a  non- 
FPRDC.  In  addition#  detailed  criteria  for  appropriate  use  of  the 
MITRE  C3i  FFRDC  is  also  contained  within  the  two  Sponsors 
operating  procedures  (ESD  Regulation  80-1  and  CECOM  Regulation  70- 
64)#  both  of  which  were  referenced  in  the  Comprehensive  review. 

In  addition#  page  3  of  the  comprehensive  review  states  that 
"At  the  first  Joint  User's  Committee  meeting  held  on  July  19# 

1990#  the  continued  need  to  DoD  of  having  MITRE 's  C3l  FFRDC  was 
reviewed  in  detail.  The  requirements  for  each  DoD  user  were 
briefed#  but  less  than  half  of  the  additional  manpower  requests 
were  recommended  for  approval  (i.e.#  only  90  of  the  140  new 
requests  were  approved)."  Prom  this  fact  alone#  how  could  the 
joint  Users  Committee  reject  so  many  programs  unless  they  had  a 
statement  that  clearly  differentiate  between  work  appropriate  for 
MITRE? 


ISSUE  OH  Page  88s  ANALYSIS  OF  THE  1990  REALIGNMENT 

The  third  sentence  reads;  "No  market  survey  was  performed  to 
establish  that  alternate  sources#  including  in-house 
personnel#  could  not  perform  the  procurement  requirement." 

OASD(C3l)  Responses  This  statement  is  not  supported  by 
requirements  contained  within  the  OPPP  Policy  Letter  84-1  or 
the  PAR  or  facts  included  within  the  Comprehensive  Review. 

As  already  stated  in  our  earlier  comments  on  this  report  and 
in  the  Comprehensive  Review#  an  informal  market  survey  was 
performed  on  each  project  and  a  positive  determination  was  made 
that  no  other  source  could  adequately  perform  the  procurement 
requirement. 


ISSUE  ON  PAGE  88s  ANALYSIS  OP  THE  1990  REALIGNMENT 

The  fourth  sentence  reads s  "Requirements  were  not  synopsized 
in  the  Commerce  Business  Daily." 

OASD(C3l)  Responses  This  statement  is  not  supported  by 
requirements  contained  within  the  OPPP  Policy  Letter  84-1  or 
the  FAR  or  facts  included  within  the  Comprehensive  Review. 
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ISSUE  ON  PAGE  S8<  ANALYSIS  OP  THE  1990  REALICNNENT 

The  fifth  sentence  reads:  "Also,  the  cost  effectiveness  of 
MITRE  perfornance  versus  non-MITRE  sources  was  not  formally 
documented  a  ** 

0ASD(C31)  Responses  This  statement  is  not  supported  by  the 
facts  included  within  the  Comprehensive  Review, 

The  Comprehensive  Review  and  the  documents/meetings/program 
reviews  it  referenced  adequately  established  that  the  MITRE  C3l 
FFRDC  was  the  only  source  DoD  had  to  receive  the  required  services 
and  that  these  services  were  cost  effective,  Xn  addition  to  the 
conclusions  made  by  the  Primary  Sponsor  and  Sponsors  based  on 
monthly  program  reviews#  the  members  of  the  Joint  Oser*s  Committee 
also  concluded  in  each  of  their  three  meetings  that  DoO  continued 
to  have  a  need  for  this  FFRDC  and  that  the  services  they  provided 
were  cost  effective  9  as  documented  in  the  Minutes  of  the  November 
19f  1992  Joint  User's  committee  meeting  (a  copy  of  these  Minutes 
was  provided  to  your  staff). 


ISSUE  ON  PAGE  88t  ANADYSIS  OF  THE  1990  REALIGNMENT 

The  sixth  sentence  reads t  "The  comments  regarding  operational 
efficiency  and  effectiveness  were  mostly  restatements  of  the 
FFRDC' s  special  relationship  and  were  not  based  on  the 
evaluation  of  specific  taskings  accomplished  by  the  MITRE  C^i 
FFRDC." 

OASD(C)I)  Response!  This  statement  is  not  accurate  and  simply 
not  supported  by  the  facts  included  within  the  Comprehensive 
Review, 

We  strongly  disagree  with  your  position  that  the  efficiency 
and  effectiveness  words  used  within  the  Comprehensive  review  were 
mostly  restatements  of  the  FFRDC 's  special  relationship  and  were 
not  based  on  the  evaluation  of  specific  taskings  accomplished  by 
the  MITRE  c3l  FFRDC,  Just  the  opposite  is  true.  Page  4  and  5  of 
the  Comprehensive  Review  included  an  "Assessment  of  the  Efficiency 
and  Effectiveness  of  MITRE'S  C3I  FFRDC  in  Meeting  DoD's  Needs," 

As  stated#  the  evaluation  addressed  technical  performance  and 
quality  of  all  contract  deliverables#  and  included  other  factors 
such  as  their  application  and  use#  and  responsiveness  and  cost 
control  management  initiatives  of  the  FFRDC,  The  Comprehensive 
Review  concluded  that  the  analyses  DoD  received  "under  the  prior 
contracts  with  NITRE'S  C3i  FFRDC  have  been  timely#  concise  and 
clear.  This  was  the  general  consensus  of  the  attendees  at  the 
July  19#  1990  Joint  Users  Committee  meeting," 

In  addition#  the  Comprehensive  review  stated  that!  "Each 
project  manager  for  each  NITRE  task  is  required  to  complete  an 
annual  MITRE  Performance  Evaluation  report  on  AFSC  1416,"  Results 
of  the  FY  1990  completed  evaluation  forms  were  assessed  by  the 
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OASDCC^Z)  and  used  as  one  of  the  factors  in  preparing  this  section 
of  the  Comprehensive  Review. 

ISSOB  ON  PAGE  8Bs  ANALYSIS  OF  THB  1990  REALIGNMENT 

The  seventh  sentence  reads i  "Ho  minutes  were  prepared  to 
document  the  reviews  and  decisions  of  the  Joint  Osers 
Committee  at  its  July  19#  1990  meeting  as  to  the  need  for  and 
effectiveness  of  the  MITRE  C3I  PPRDC." 

OASD(C3i)  Responses  This  is  a  misleading  statement. 

Even  though  no  formal  document  called  "MINUTES"  was  prepared 
for  the  July  19#  1990#  meeting#  the  AS0(C3l)  signed  a  memorandum 
on  August  6#  1990#  that  summarised  the  decisions  made  during  the 
Joint  Users  Committee  (JUC)  meeting.  This  memorandum  and  copies 
of  the  viewgraphs  used  during  the  meeting  documented  the  reviews 
and  decisions  of  the  JUC  members  as  to  the  need  for  and 
effectiveness  of  the  PPRDC.  In  addition#  the  ASU(C3I)  memorandum 
dated  August  6#  1990  stated i  "Because  of  ceiling  growth 
constraints#  it  was  not  possible  to  accommodate  all  of  the 
requested  increases  for  PY  1991  MITRE  Support."  As  already 
stated#  90  MTS  requests  were  not  recommended  by  the  JUC  members 
nor  approved  by  the  Primary  Sponsor  [ASD(C3l}]« 


ISSUE  ON  PAGE  882  ANALYSIS  OF  THB  1990  REALIGNMENT 

The  eighth  sentence  reads:  "The  review  did  not  discuss  the 
controls  established  by  NITRE  to  assure  cost  effective 
operations." 

0ASD(C3I)  Response:  This  is  a  misleading  statement. 

Page  5  of  the  Comprehensive  review  included  a  review  on  the 
"Assessment  of  the  Adequacy  of  the  MITRE  C3Z  PPRDC  Management  in 
Ensuring  a  Cost  Effective  Operation."  The  Primary  Sponsor's 
review  in  these  areas  examined  numerous  items#  including  an 
independent  assessment  on  the  quality  and  use  of  all  FY  1990 
contract  deliverables.  The  goal  was  to  measure  their  value  and 
use#  and  thus  be  in  a  position  to  evaluate  their  cost 
effectiveness.  This  assessment  was  briefed  to  the  Joint  Users 
Committee.  This  briefing  and  other  items  they  considered# 
provided  more  than  an  ample  basis  that  allowed  them  to  make  the 
conclusion  that  NITRE  services  were  cost-effective. 


ISSUE  ON  PAGE  88:  ANALYSIS  OF  THB  1990  REALIGNMENT 

The  ninth  (and  last)  sentence  reads i  Moreover#  the  review  did 
not  adequately  address  whether  the  PAR  criteria  for 
establishing  the  PPRDC  continue  to  be  met  or  that  the 
sponsoring  agreement  complied  with  the  FAR." 

0ASD(C31)  Response:  This  statement  is  simply  not  correct. 
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DEPUTY  ASSISTANT  SECRETARY  OP  DEFENSE  (STRATEGIC  AND  TACTICAL  C3) 
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TAB  C 


As  stated  on  page  one  of  the  Comprehensive  Review^  the  Under 
Secretary  of  Defense  for  Acquisition  by  memoranduoi  dated  February 
2I0  1990#  designated  the  OASO(C3l)  as  the  new  Primary  Sponsor  of 
MITR£*s  C^l  FFRDCt  There  was  no  Sponsoring  Agreement  that  covered 
the  new  Primary  Sponsor  prior  to  this  change  in  sponsorship. 
Accordingly#  it  was  impossible  to  certify  that  an  existing 
Sponsoring  Agreement  complied  with  the  FAR. 

Page  one  further  states  that  OFPP  Policy  Letter  B4-1  and  the 
FAR  require  the  Primary  Sponsor  to  conduct  a  comprehensive  review 
before  a  contract  or  Sponsoring  Agreement  is  renewed.  Even  though 
the  Primary  Sponsor  would  never  have  a  contract  with  the  FFRDC 
(the  Army  contract  was  nothing  more  than  a  division  of  an  existing 
DoD/Air  Force  contract)#  the  OFPP  Policy  Letter  84*1  and  the  FAR 
still  require  the  Primary  Sponsor  to  have  a  Sponsoring  Agreement. 
Accordingly#  both  a  Comprehensive  Review  and  a  new  Sponsoring 
Agreement  were  prepared.  As  stated  on  page  one#  the  new 
Sponsoring  Agreement  was  attached  to  the  Comprehensive  Review  and 
specifically  stated  ...  "It  is  now  ready  for  signature  by  the 
AS0(C3l)  and  the  President  of  MITRE."  When  the  ASD(C3l)  signed  the 
Sponsoring  Agreement#  he  simultaneously  approved  the  new 
Comprehensive  Review. 

Pages  5  and  6  of  the  Comprehensive  Review  further  states  that 
the  new  MITRE  C^I  FFRDC  Sponsoring  Agreement  was  fully  responsive 
to  all  of  the  requirements  contained  within  both  the  OFPP  Policy 
Letter  84-1  and  Section  35.017*4  of  the  FAR.  Therefore#  to  imply 
that  the  Comprehensive  Review  did  not  address  whether  or  not  the 
Sponsoring  Agreement  complied  with  the  FAR  is  incorrect. 

With  regard  to  addressing  the  criteria  for  establishing  that 
the  FFRDC  continues  to  exist#  your  attention  is  directed  to  page  2 
of  the  Comprehensive  Review  which  states 1 

"The  basic  nature  of  MITRE'S  C3l  FFRDC  support  to  DoD  users 
has  not  changed  since  the  last  reviews  -  October  1#  1985#  and 
October  1#  1988.  The  February  21#  1990#  change  in  sponsorship 
did  not  change  the  FFRDC's  purpose#  mission#  and  general  scope 
of  effort#  which  continues  to  be  to  perform  DoD-wide  C3l 
systems  engineering.  • 

This  is  the  same  criteria  used  to  establish  the  FFRDC  in  1958#  and 
the  Comprehensive  Review  determined  that  this  criteria  continues 
to  be  satisfied.  Specifically#  page  5  of  the  Comprehensive  review 
states:  "After  detailed  examination  and  review#  the  0ASD(C3i)  has 
concluded  that  all  of  the  requirements  for  using  MITRE 's  C3l  FFRDC 
as  listed  in  the  OFPP  Policy  Letter  84-1  and  the  FAR  are  being 
met.” 
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OFFICE  OF  THE  AS8I8TAHT  SECRETARY  OF  DEFENSE  (PRODUCTION  AND 
LOGISTICS)  COMMENTS 


THE  OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 
TMNEC  SKVUNE  FtACE.  SUITE  1100 
S201  UESSURO  FIKE 
FALLS  CHURCH,  VIROINIA  22041 -3203 


fSOOUCnOMAND  July  26.  199) 

LOOISTICt 

MSC 


MEMORANDUM  FOR  DIRECTOR*  CONTRACT  MANAGEMENT*  OFFICE  OF  THE  000 
INSPECTTm  GENERAL 

SUBJECT:  Draft  Audit  Report  on  Sole-Source  Justification  for 

OoD-Sponsored  Federally  Funded  Research  and  Development 
Centers  (FFRDCs);  Project  Ho.  lCH-5012.02 


He  have  reviewed  the  draft  audit  report  transndtted  by  your 
memorandum  of  March  23*  1993.  Our  understanding  was  that  the  0DDR6E 
reply  would  represent  the  OSD  response*  since  they  manage  FFRDC 
Programs*  and  certainly  would  represent  the  OUSD(A)  response*  of 
which  the  A5D(PCL)  is  a  part.  Therefore*  our  response  is  subsumed 
within  the  DDRCB  reply.  He  agree  with  their  comments. 

Please  call  Ms.  Kimble  Pendley  at  756-2200  if  you  need  further 
assistance. 

Director 

Management  Support  Center 


Kichard  E. 
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DEPARTMENT  OF  THE  ARMY  COMMENTS 


DEPARTMEMT  OF  THE  ARMY 
AOIMHlSTtUTIVC  ASSISTANT  TO  THE  SECNCTAIIY 
WASHINOTON.  OjC. 

17  May  1993 


MEMORANDUM  FOR  THE  INSPECTOR  GENERAt,  DEPARTMENT  OF  DEFENSE 
(AUDITING) 

SUBJECT:  Inspector  General,  Department  of  Defense,  Draft  Report 
on  Sole  Source  Justifications  for  DoD-Sponsored 
Federally  Funded  Research  and  Development  Centers 
(Project  No.  1CH-5012.02) 


This  office  has  reviewed  the  subject  draft  report  and 
comments  are  being  provided  as  requested.  Recommendation  2  was 
the  only  recommendation  addressed  to  us,  however,  we  believe 
comments  are  warranted  on  the  resiaining  recommendations  and 
therefore  are  being  provided. 

Milton  H.  Hamilton 


Attachment 
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finding 

The  DoD  sponsors  did  not  adequately  document  the  basis  for 
renewing  the  ten  FFRDC  sole*source  contracts  we  reviewed.  The 
DbD  sponsor  comprehensive  reviews  did  not  meet  OFPP  Policy  Letter 
84-1  and  FAR  35.017-4  requirements,  and  the  sole-source 
justifications  for  renewal  of  the  FFROC  contracts  contained 
unproven  statements.  Three  reviews  were  not  done,  six  reviews 
were  done  after  approval  of  the  sole-source  justifications  and 
were  not  adequate,  and  one  review  was  incomplete.  The  inadequate 
reviews  and  sole-source  justifications  occurred  because  senior 
officials  in  the  OSD,  the  Military  Departments,  apd  the  Defense 
Advanced  Research  Projects  Agency,  and  the  responsible 
contracting  officers  did  not  challenge  the  timeliness, 
completeness,  and  accuracy  of  the  comprehensive  reviews  and  sole- 
source  justifications.  The  DoD  FFROC  Management  Plan,  which  is 
being  implemented,  does  not  go  far  enough  to  correct  the  review 
and  justification  documentation  problems.  Thus,  the  reasons  used 
to  justify  the  sole-source  contracts  for  the  ten  FFRDCs  were  not 
valid.  AS  a  result,  the  sole-source  justifications  restricted 
competition  from  private-sector  firms  and  probably  resulted  in 
higher  costs  for  performance  of  some  work.  Also,  the  Navy  can 
save  $7.8  million  over  two  years  by  replacing  Center  for  Naval 
Analyses  field  analysts  with  Navy  employees. 


Additional  Factsi 

The  assumption  underlying  recommendation  1.  a.  is  that  DSS-N  did 
not  follow  the  regulations,  thus  making  award  of  the  contracts 
invalid.  The  requirement  for  conducting  a  comprehensive  review, 
set  forth  in  FAR  35.017  dated  March  1990,  post-dated  all 
currently  existing  contracts  our  agency  has  with  FFRDCs,  with  the 
exception  of  MDA903-91-C-0006  with  the  Arroyo  Center,  awarded  in 
October  1990.  A  comprehensive  review  in  accordance  with  FAR 
35.017-4  was  performed  prior  to  award  of  that  contract  and  this 
review  is  considered  acceptable.  The  reviews  performed  on  the 
other  contracts  conformed  with  OFPP  Circular  84-1,  the  cmly 
regulatory  guidance  then  available.  We  do  not  believe  that  any 
comprehensive  reviews  on  FFRDC  contracts  should  be  required  until 
the  current  contracts  expire.  All  existing  regulations  were 
followed  at  time  of  award.  Thus  there  is  no  legitimate  legal 
basis  for  conduct  of  such  reviews.  Clearly,  initiating  such 
reviews  at  this  stage  in  effect  stops  or  significantly  delays 
future  work  and  would  be  unfair  to  the  FFRDCs  whose  planning  and 
staffing  would  be  seriously  disrupted.  Such  action  also  could 
adversely  affect  efforts  required  to  deal  with  significant 
defense  issues  on  a  timely  basis. 
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Zn  reQards  to  recoaunendfttion  1.  b.  (2),  we  caution  that 
considering  additional  in-house  staffing,  in  lieu  of  contracting 
with  an  FFRDC,  or  any  outside  business  concern,  nay  not  be  a 
viable  alternative.  With  the  Military  and  civilian  drawdowns 
continuing  and  even  accelerating,  the  likelihood  of  obtaining 
additional  in-house  Manpower  to  support  the  kinds  of  studies  the 
sponsors  need  appears  far  from  promising. 

DSS-H  has  serious  reservations  about  recomnendation  1.  c»  for 
requiring  a  Commerce  Business  Daily  announcement  of  intent  unless 
the  regulations  are  revised  to  specifically  provide  for  It. 
Although  a  Market  survey  of  this  type  is  one  tool  which  may  be 
used,  it  is  not  the  only  one.  While  FAR  3S. 017-4  provides  that 
the  requlreaents  for  a  comprehensive  review  must  include, 
"consideration  of  alternative  sources  to  meet  the  sponsors 
needs,"  the  section,  in  not  specifying  how  consideration  of 
alternative  sources  is  to  be  conducted,  provides  contracting  and 
program  officials  considerable  latitude  on  how  they  will  perform 
this  function. 

POD  IG  Reconunendation  2 

We  recommend  that  the  Director,  Defense  Research  and  Engineering; 
the  Assistant  Secretary  of  Defense  (Production  and  Logistics); 
the  Assistant  Secretary  of  the  Wavy  (Research,  Development,  and 
Acquisition);  the  Assistant  Secretary  of  the  Air  Force 
(Acquisition);  and  the  Director,  Defense  Supply  Servlce- 
Washington  not  award  any  contracts  or  exercise  any  options  or 
modifications  to  existing  contracts  pending  the  completion  of  an 
acceptable  comprehensive  review  and  a  sound  sole-source 
justification  for  the  applicable  FFRDC. 


DSS-W  Comments 

Nonconcur.  As  we  indicated  in  response  to  recommendation  1.  a. 
above,  award  of  all  of  our  FFRDC  contracts  was  in  accordance  with 
existing  requirements.  Requiring  a  comprehensive  review  in  the 
middle  of  a  contract,  now,  when  it  was  not  originally  required 
constitutes  drastically  changing  the  rules  and  is  not  consistent 
with  acceptable  standard  business  practice.  As  noted  previously, 
the  contract  for  the  Arroyo  Center  was  awarded  after  completion 
of  a  comprehensive  review  on  7  June  1990,  and  should  thus  not 
require  an  additional  review  unless  a  new  contract  is 
contemplated . 
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DEPARTMENT  OP  THE  NAVY  COMMENTS 


DEPARTMENT  OF  THE  NAVY 

OFnC£  OF  THE  ASSISTANT  SECftETARY 
(RMMfch.  D*v«lopn>«nl  tnd  AcquitAioo) 
WASHINGTON,  DC  20360^1000 

V9  m  1993 


MEMORANDUM  FOR  THE  DIRECTOR,  DEPARTMENT  OF  DEFENSE  INSPECTOR 

GENERAL  CONTRACT  MANAGEMENT  DIRECTORATE 

Subj:  DRAFT  AUDIT  REPORT  ON  SOLE-SOURCE  JUSTIFICATIONS  FOR 

DOD-SPONSORED  FEDERALLY  FUNDED  RESEARCH  AND  DEVELOPMENT 
CENTERS  (PROJECT  NO.  lCH-5012.02)  -  ACTION  MEMORANDUM 

I  am  responding  to  the  draft  audit  report  forwarded  by  Tab  A 
concerning  the  justifications  for  sole-source  contract  awards  to 
Federally  Funded  Research  and  Development  Centers  (FFRDCs) • 

Tab  B  is  the  Department  of  the  Navy's  (DON'S)  response  to 
the  recommendations  contained  in  the  report.  Comments  regarding 
internal  controls  raised  in  the  introduction  (Part  I  of  the 
report)  are  discussed  at  Tab  C  and  comments  regarding  the 
findings  raised  in  Part  II  are  at  Tab  D*  Tab  E  addresses  the 
specific  details  of  the  contract  with  the  Center  for  Naval 
Analyses  (CNA)  contained  in  Appendix  J.  The  comments  contained 
in  Tabs  B  through  E  relate  only  to  FFRDCs  in  general,  and  CNA  in 
particular.  They  do  not  address  issues  that  pertain  to  FFRDCs 
that  are  not  sponsored  by  the  DON* 

Our  response  reflects  the  experience  and  frustration  of  Navy 
managers  and  "customers’*  interviewed  by  the  IG  auditors  during 
this  effort.  It  remains  apparent  that  your  auditors  have  not 
only  failed  to  understand  the  need  for  our  only  FFRDC  and  how  we 
manage  it,  but  also  little  heeded  our  attempts  to  explain. 


Edward  C.  Whitman 


Copy  to: 

HAVINSGF.N 

HCB-53 


Tab  A 

Tab  B 

Tab  C 

Tab  D 
Tab  E 


DODIG  Draft  Audit  Report  of  23  Mar  93  Project  No. 
lCH-5012.02 

DON  Response  to  DODIG  Draft  Audit  Report 
(Recommendations) 

DON  Response  to  DODIG  Draft  Audit  Report  (Internal 
Controls) 

DON  Response  to  DODIG  Draft  Audit  Report  (Findings) 
DON  Response  to  DODIG  Draft  Audit  Report  (Appendix  J) 
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Final  Report 


Revised 
PP.  16  and  17 


Department  of  the  Navy  Response 
to 

DODIG  Draft  Audit  Report  of  March  23,  lj^3 
on 

Sole-source  Justification  for  DOD-sponsored 
Federally  Funded  Research  and  Development  Centers 
Project  No.  lCH-5012.02 
(Recommendations) 

RECOMMENDATION  la; 

We  recommeod  Uuit  Iht  Director.  Defease  Resesrtb  tod  Eagineerinf  (DDR&E)  direct  DOD  sponsors  for  all 

FFRDCs  to  prepare  new  oompreheosive  reviews. 

DON  Position!  Nonconcur.  The  comprehensive  review  for  renewal  of  CNA’s 
current  contract  was  prepared  in  accordance  with  the  guidance  in  OFPP  Policy 
Letter  84-1  and  FAR  35.017-4(c}  and  addressed  the  requirements  of  both. 

Further,  since  it  can  take  up  to  one  year  to  complete  a  thorough  comprehensive 
review,  and  since  the  next  comprehensive  review  of  CNA  is  intended  to  begu 
during  the  last  quarter  of  fiscal  year  1994,  it  would  be  an  imprudent  use  of  limited 
resources  to  conduct  such  a  review  at  this  juncture.  We  are,  however,  interested 
in  ways  to  improve  the  review  process  and  will  consider  the  recommendations  of 
die  auditors  in  conducting  the  next  comprehensive  tevkw. 

RECOMMENDATION  lb; 

We  lerommmd  that  the  DDR&E  require  the  DOD  ipoiison  for  ell  FFRDCt  to  documeol  in  the  reviewK 

•  The  fecial  rtseudi  end  development  needs,  the  special  tfcilltend/orci^bililiec  involved  in 

meeting  the  needs,  end  the  estimated  extent  of  the  needs. 

-  Tbe  tlleniative  for  accomptidung  Ibe  lequirements.  m  incliide  edditiooil  in-bouse  ataffing. 
oooventtooal  cootiacts.  and  other  DOD  and  noo-DOD  FFRDCb.  Tbe  teview  sboutd  eoo^m  tbe 
ooet  to  obtain  support  from  Ibe  FFRDC  versus  tbe  cost  to  perfonn  tbe  work  with  additiond 
Government  persoo2»d. 

•  The  otjective  assessment  of  FFRDC  cfficicocy  and  effectivcoess  in  meeting  DOD  sponsor  needs, 
to  include  Ibe  number  and  dollar  value  of  projects  and  pfograms  afrertfid  and  followmp  and 
resolution  by  bigber-level  managcmenl  of  all  project  and  program  assessments  that  were  critical  of 
tbe  FFRDC. 

•  Tbe  controls  esubitshed  by  tbe  FFRDC  to  ensure  a  cost<cflective  operation.  At  a  mmiiDum, 
reviews  should  address  the  maintenance  of  accurate  accounting  records,  tbe  following  of  loiiid 
procuremeol  practioes,  Ibe  leasootbleness  of  overhead  costs,  and  ibe  need  for  management  fees. 

-  Tbe  assurances  that  criteria  for  esUblishing  an  FFRDC  continue  to  be  satisfied  and  that  an 
adequate  sponsoring  agreement  exists. 

PON  Eflsilion:  Partiany  concur.  While  we  agree  wth  the  intent  of  ihe  auditors 
to  ensure  the  requirements  of  OFPP  Policy  Letter  84-1  and  FAR  35.017-4(c)  are 

TABB 
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met,  we  do  not  agree  with  the  reconunendalion  that  DDR&E  should  impose 
additional  requirements  for  conducting  comprehensive  reviews.  Additionally, 

FAR  3S.017-4(a)  clearly  places  the  lesponsibili^  for  conducting  comprehensive 
reviews  with  the  primary  sponsor.  It  is  the  responsibility  of  the  sponsor,  not 
DDR&E,  to  determine  the  proper  methodology  and  documentation  for  the  review 
process.  However,  as  stated  above,  the  DON  will  consider  the  reconunendadons 
of  the  auditors  in  conducdng  its  next  comprehensive  review  of  CNA. 

PFrOMMENDATION  let 

We  (tcomineod  Ibal  the  DDR&E  require  the  tponsprs  to  puMish  edvtnce  notices  of  intent  in  the  Commerce 
HiLiiness  Dsilv  end  to  perform  nmtiet  surveys  to  vslidste  thst  ■  sole-source  awerd  is  warranted  because 
comp^tion  is  infeasible. 

DON  Position!  Nonconcur.  This  recommendation  is  inconsistent  with  current 
law  or  regulation  and  is  without  merit.  FAR  6.302-^3(a)(2),  under  the  authority  of 
41  U.S.C.  253(cX3)  and  10  U.S.C.  2304(c)(3),  specifically  allows  FFRDCs  to  be 
exempted  from  fall  and  open  competition  and  thus,  under  FAR  5.20)(a)(10)  (by 
authority  of  41  U.S.G.  416(c)(2)),  there  is  no  requirement  to  publish  advance 
notices  of  proposed  contract  actions  in  the  Commerce  Business  Daily  (CBD).  The 
deliberate  references  to  FFRDCs  in  the  aforementioned  FAR  and  statute  citations 
unquestionably  demonstrate  that,  as  a  matter  of  public  policy  and  in  the  national 
interest,  FFRDCs  are  exempted  from  normal  competitive  procurement  actions 
(including  publication  of  advance  notices).  Such  intentional  protection  has  been 
instituted  in  clear  recognition  that  FFRDCs  are  valuable  and  unique  national  assets 
that  (as  stated  in  FAR  3S.017(a)(2)): 

•...(meet]  some  special  long-term  research  or  development  need  which 
cannot  be  met  as  effectively  as  by  existing  in-house  or  contractor 
resources.  FFRDCs  enable  agencies  to  use  private  sector  resources  to 
accomplish  tasks  that  are  integral  to  the  mission  and  operation  of  the 
sponsoring  agency.* 

and  (as  stated  in  FAR  3S.OI7(a)(4)); 

•Long-term  relationships  between  the  Government  and  FFRDCs  are 
encouraged  in  order  to  provide  the  continuity  that  will  attract  high- 
quality  personnel  to  the  FFRDC.  This  relationslup  should  be  of  a 
type  to  encourage  the  FFRDC  to  maintain  currency  in  its  field(s)  of 
expertise,  maintain  objectivity  and  independence,  preserve  familiarity 
with  the  needs  of  its  sponsorfs),  and  provide  a  quick  response 
capability.* 
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The  very  fiid  that  this  recommendation  has  been  included  in  the  report  indicates 
that  the  auditors  have  not  luUy  grasped  the  concept  of  the  uro'que  status  of 
FFRDCs  and  their  special,  long-term  relationships  with  their  sponsors.  We 
believe  the  failure  on  the  part  of  the  auditors  to  acknowledge  and  address  this 
concept  in  the  report  represents  a  serious  flaw  of  tfte  report  itself  and  impinges 
upon  its  reliability  and  usefulness.  Each  FFRDC  must  be  exammcd  in  the 
aggregate  and  its  special  competencies  must  be  viewed  in  the  long-term  when 
considering  the  adequacy  of  sole-source  justifications  and  the  decision  not  to 
publish  advance  notices  in  the  CBD. 

recommendation  Idt 

We  tlut  DDR&E  discoDtiDue  FFROCi  dm  caonal  be  adequtely  justified. 

DON  PnsHion!  Partially  concur.  We  agree  that  organizations  that  no  longer  meet 
Ac  requirements  of  FAR  35.017  and  Aus  by  definition,  ought  to  no  longer  be 
considered  FFRDCs,  should  be  subject  to  termination  of  Aeir  sponsoring 
agreements.  However,  in  accordance  wiA  FAR  35.017-2  and  35.017-4,  a 
determination  of  this  kind  is  a  matter  within  the  purview  of  Ae  primary  sponsor 
and  DDR&E. 


mm 


ENDATION  2; 


We  lecommead  that  the...Asaistaat  Secretaiy  of  the  Navy  (Reteai^,  E)cve]opixie&t  and 
Acqui$itioo)[ASN(RD&A)];...not award  any  cootracu  or  cxerdae  any  optioos or  modificatiooi  to  eai^ 
coDtracU  peeing  the  completion  of  an  accq)tahle  conpfdtenuve  review  and  a  cound  tole-ioufce  justification 
for  the  applicable  FFRDC. 

DON  Portion!  Nonconcur.  While  we  can  agree  wiA  Ae  undeityAg  rootiwtion 
for  inclusion  of  this  recommendation  Qt  indeed  that  underlying  motivation  is  to 
induce  primary  sponsors  to  reevaluate  and  improve  upon  Ae  processes  for 
conducting  comprehensive  reviews  and  thereby  Ae  improve  Ae  basis  for  award  of 
sole-source  contracts),  we  cannot  concur,  however,  wiA  Ae  intent  of  Ae 
recommendation  to  halt  contract  modifications  and  thereby  cause  a  complete 
cessation  of  FFRDC  operations.  In  Ae  case  of  CNA,  contract  modifications 
provide  Ae  means  of  obligating  funds  to  CNA.  To  cause  Ae  discontinuation  of 
this  routine  practice  would  deprive  Ae  DON  wiA  a  source  of  indqiendent, 
unbiased,  auAoritative,  and  analytically  rigorous  research.  In  addition,  such  an 
action  b  completely  inapproprbte  inasmuch  as  comprehensive  reviews  were 
conducted  in  good  faiA  wiA  the  full  intention  of  complying  wiA  Ae  requirements 
of  the  FAR  35.017-4  and  OFPP  Policy  Letter  84-1 .  Actions,  swA  as  Aosc 
recommended,  are  not  warranted  and  are  unnecessary. 
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We  recoaubeiid  that  the  ASN(RD^)  perform  the  required  ooct  aoalyaea  aod  prepare  the  budget  documents  to 
replace  CNA  field  aaalysts  with  less  c^y  ia-bouse  personocl. 

DON  Position!  Nonconcur,  It  appears  that  the  sole  criterion  used  by  the  auditors 
in  their  examination  of  the  CNA  field  program  is  that  of  cost.  We  believe  that 
limiting  the  examination  to  only  the  salary  and  benefits  costs  of  individual  field 
analysts  reflects  a  severe  lack  of  understanding  of  the  role  and  function  of  the 
CNA  field  program  and  its  benefit  to  the  DON.  One  of  the  primary  reasons  CNA 
exists  as  an  FFRDC  is  because  its  field  constitutes  one  of  its  long*tenn 

special  competencies  and  capabilities  that  c^ot  be  met  effecdvely  by  i|i-house 
or  other  contractor  resources.  Along  with  providing  direct  analytic^  support^ 

CNA  analysts,  through  CNA’s  world-wide  field  program,  obtain  arul/or  refresh 
their  in-depth,  hand-on,  familiarity  with  fleet  operations  of  the  Navy  and  Marine 
Corps.  It  is  singularly  relevant  that  CNA  analysts  must  possess  a  thorough 
understanding  of  this  unique  nature  of  the  Navy  and  Marine  Corps,  the  only 
military  services  that  deploy  to  sea,  in  order  to  provide  the  high  quality  research 
products  the  DON  demands. 

The  direct  benefit  to  the  DON  of  CNA*$  field  program  is  two-fold.  First,  >i4iilc 
serving  with  the  fleet,  CNA  field  analysts  (the  majority  of  whom  possess  doctoral 
degrees)  are  a  source  of  on-site  independent,  immediate  (or  quick  response) 
analytic^  support  of  fleet  and  field  exercises;  actual  operations,  such  as  Desert 
Shield/Storni  and  Restore  Hope;  and  evaluation  of  operational  capabilities  and 
tactics.  Second,  due  to  the  operational  experience  of  CNA  Washington  analysts 
who  have  completed  one  or  more  field  representative  “tours",  CNA’s 
headquarters-based  research  program  is  imbued  with  a  degree  of  operational 
realism  that  is  unique  among  FFRDCs. 

In  addition  to  the  qualitative  aspects  of  the  field  program  dial  have  been  neglected 
by  the  report,  the  auditors  apparently  did  not  give  full  consideration  to  die 
state-of-the-art  infrastructure  CNA  headquarters  provides  in  support  of  its  field 
program.  The  report  does  not  address  the  added  cost  of  replacing  or  duplicating 
this  necessary  intotructure,  nor  does  it  address  the  added  value  of  current 
research  utilizing  the  available  headquarters  infrastructure. 

Even  a  perfunctory  examination  of  replacement  staffing  for  CNA  would  show  dial 
the  number  of  personnel  who  possesses  the  requisite  educational  background  and 
expertise  are  not  available  to,  nor  are  apt  to  be  recruited  by,  the  DON.  Further,  it 
would  not  be  cost  effective  for  the  DON  to  attempt  to  replace  CNA’s 
infrastructure  with  its  own.  It  is  therefore,  extremely  unlikely  that  the  DON 
would  achieve  significant  economies  by  replacing  CNA  with  DON  resources.  In 
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fact,  it  is  moie  likely  that  negligible  economies  would  be  achieved,  and  a 
significant  degradation  of  research  independence  and  caliber  would  resuh. 
Replacentent  of  the  CNA  field  program  with  in-house  resources,  as  is  suggested  by 
this  recommendation,  is  moot.  In  today’s  era  of  downsizing,  the  DON  does  not 
have  the  billets  nor  the  infrastructure  to  replace  CNA. 


Deleted 


We  fecommeod  Ihit  the  Director  of  Defease  Procurenieot  direct  the  Defease  Ac<)uisitioo  Regulstioos  Couacil  to 
add  provisions  to  the  Defense  Federal  Acquisitioo  Regulation  Supplement  [DFARS]  to  require  rtaewal  of 
cootracu  with  DOD  FFRDCs  to  be  publicized  in  advance.  App^tx  Q  (of  the  draft  audit  rqK>rt]  shows  the 
pfopoead  changes  to  the  DFARS.  Qf  the  Director  desires  to  propose  a  nvisioo  to  the  Federal  Acquisitioo 
Relation  (FAR)  to  satisfy  the  intent  of  the  lecommendatioo,  we  have  included  language  in  Appendix  Q  to 
revise  FAR  5.202(aK]0)  and  FAR  S.205(b)). 

DON  Position!  Nonconcur.  As  stated  in  our  response  to  Recommendation  Ic, 
there  rests  within  the  FAR  specific  authority  to  exempt  FFRDCs  from  the 
requirements  of  full  and  open  competition.  This  authority  is  derived  from  die 
statutory  authority  contained  in  41  U.S.C.  2S3  and  10  U.S.C<  2304,  aiKl 
41  U.S.C.  416,  which  in  itself  demonstrates  the  intent  of  the  Congress  regarding 
FFRDCs.  Wc  believe,  therefore,  that  the  report,  dirough  this  rcconunendation, 
goes  beyond  the  scope  of  the  audit  and  enters  into  the  realm  of  legislative  powers 
that  are  Constitutiofudly  reserved  for  the  Congress. 
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Federally  Funded  Research  and  Development  Centers 
Project  No.  lCH-5012.02 
(Internal  Controls) 


intcrnalc 


ILS(PG4  0FTHEREP0RT): 


The  iiiidit  tdeatified  a  miterial  iniemal  cooMol  weakness  as  dcfiaed  by  'Public  Law  9^-225  |97-2SS],  Office  of 
MaikiieiDeat  and  Budsef  Ciicular  A-123,  add  DOD  Directive  5010.31.*  laletnal  adaimitralive  cooirols  were 
not  adequate  to  verify  that  oomptebeosive  leviews  of  FFRDCs  met  the  lequiteaent  of  the  FAR  and  that 
justifications  for  sole-source  coetrscts  for  FFRDCs  were  prdperiy  support^.  Spoosoriaf  ptogiam  officials  and 
aeoior  acquisition  officials  fiuled  to  exercise  their  supervisory  responsibilities  and  ooolractias  officers  failed  to 
adhere  to  established  policies  and  procedures  when  renewing  sole-source  cootracts  for  FFRDCs. 


DON  Position:  Nonconcur.  In  the  case  of  CNA,  we  most  emphatically  object  to 
the  auditdrs*  contention  that  sponsoring  program  officials  and  senior  acquisition 
officials  failed  to  exercise  their  supervisory  responsibilities  and  contracting  officers 
failed  to  adhere  to  established  policies  and  prex^ures  when  renewing  die  sole- 
source  contract  with  CNA.  The  process  of  renewing  the  CNA  contract  took  place 
over  a  period  of  more  duin  six  months.  During  that  period,  program  office  and 
contracting  office  personnel  woriced  extremely  closely  to  ensure  all  regulatory 
requirements  for  contract  renewal  were  met.  Staff  personnel  were  in  constant 
contact  and  were  thoroughly  familiar  with  aB  aspects  of  the  contract  renewal 
process.  Additionally,  they  ensured  senior  acquisition  officials  (PDASN(RDAA) 
and  ASN(RD&A))  were  kept  apprised  (through  periodic  verbal  briefings)  of  die 
progress  of  renew^  of  CNA‘s  contact.  Officials  were  aware  of  dieir 
responsibilities  and  carried  them  out  to  the  best  of  their  abiUties. 


This  contention  of  material  internal  control  weakness  appears  to  rest  upon  the 
auditors*  opinion  that  comprehensive  reviews  were  inadequate.  As  staled  in  our 
response  to  Recommendation  la,  the  CNA  comprehensive  review  was  prepared  in 
compliance  with  the  requirements  of  FAR  3S.017-4  and  OFPP  Policy  Letter  84-1 
in  good  faith.  To  ensure  such  compliance,  we  consulted  with  the  Director  of 
FFRDC  Programs  at  DDR&E  prior  to  conunencing  the  review.  He  provided  a 
suggested  mediod  for  conducting  the  review,  which  we  adopted.  Although  the 
memorandum  promulgating  the  results  of  the  review  was  not  actually  signed  until 
after  the  contract  was  awarded,  all  interested  parties  were  apprised  of  its  results 
prior  to  the  award.  The  contracting  officer  and  the  senior  acquisition  official  had. 
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prior  to  28  September  1990,  satisfied  Aemselves  that  there  existed  sound  and 
justifiable  grounds  for  renewing  CNA’s  contract  on  a  sole-source  basis. 

We  most  strongly  object  to  the  use  of  the  term  'material  weakness',  which  is 
defined  in  0MB  Circular  A-123  as: 

"a  specific  instance  of  non-compliance  with  the  Integrity  Act  of 
sufficient  importance  to  be  reported  to  the  President  and  Congress. 

Such  weakness  would  significantly  impair  the  fulfillment  of  an  agency 
component’s  mission;  deprive  the  public  of  needed  services;  violate 
statutory  or  regulatory  requirements;  significantly  wet^en  safeguards 
against  waste,  loss,  unauthorized  use  or  misapprq>riation  of  funds, 
property,  or  other  assets;  or  result  in  a  conflict  of  interest." 

Surely,  any  negative  findings  of  this  report  are  not  of  such  severity  as  neMssitate 
invoking  the  term  'material  weakness’  with  all  its  accompanying  implications. 
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to 

DODIG  Draft  Audit  Report  of  March  23, 1993 
on 

Sole^source  Justification  for  DOD-sponsored 
Federally  Funded  Research  and  Development  Centers 
Project  No.  lCH-5012.02 
(Findings) 

FINDING  It  BASIS  FOR  SOLE-SOURCE  CONTRACT  RENEWALS  (PG  Ii)F 
THE  REPORT.  1ST  PARAGRAPH); 

The  DOD  spoasors  did  not  adequstely  document  ibe  basis  for  reoewiog  the  10  FFRDC  aole-soiutc  cootracti  we 
reviewed.  Tbe  DOD  sponsor  comprdietisive  reviews  did  not  meet  OFPP  PoUcy  Letter  S4->1  and  FAK  15.017-4 
requiremeots,  and  Ibe  sole-source  Justifications  for  renewal  of  the  FFRDC  eoolracts  contained  unproven 
ilatemeots.  Three  reviews  were  not  done,  six  reviews  were  done  after  approval  of  the  sole-aouice  justifications 
and  were  not  adequate,  and  one  review  was  incomplete.  Tbe  inadequate  reviews  and  sole-source  justificationt 
occurred  because  senior  officials  in  tbe  OSD,  tbe  Military  Departments,  and  Ibe  Defense  Advanced  Reaeaicb 
Projects  Acency  [now  known  as  Advanced  Research  Projects  Agency]  and  tbe  responsible  contractiof  officers 
did  not  cballenge  the  timeliness,  completeness,  and  accuracy  of  tbe  comprehensive  reviews  and  sole-aouroe 
justifications.  Tbe  reasons  used  to  justify  the  sole-source  contracts  for  the  10  FFRDCs  were  not  valid. 

DON  Position!  Nonconcur.  This  finding  appears  to  be  based  upon  an  editorial 
opinion  of  the  documentation  of  the  comprehensive  review  process.  As  we  have 
previously  stated,  the  DON  is  interested  in  improving  the  review  process, 
including  documentation.  However,  whatever  die  auditors*  opinion  of  die  \vrite- 
up,'  the  comprehensive  review  was  developed  and  conducted  in  compliance  widi 
FAR  35 .017-4  and  OFPP  Policy  Letter  84-1 .  We  believe  the  review  did  provide  a 
valid  basis  for  renewal  of  CNA*s  contract  and  that  the  sole-source  justilicatioa  did 
not  contain  unproven  statements.  As  stated  in  Tab  bfficiab  acted  property  in 
carrying  out  dieir  responsibility  regarding  renewal  of  CNA*s  contract  and  that 
diere  were,  in  ftict,  valid  reasons  to  justify  award  of  a  sole-source  contract  to 
CNA. 


iG  2;  BASIS  FOR  SOLE-SOURCE  CONTRACT  RENEWALS;  REPLACING 

CNA  FIELD  ANALYSTS  WITH  NAVY  EMPLOYEES  (PG  7  OF  TOE  REPORT. 

1ST  PARAGRAPH.  LAST  3  LINES): 


The  Navy  can  save  $7.$  buHioq  ever  2  yean  by  replaciof  CNA  field  analyxti  with  Navy  muployeea. 


DOD  Position!  Nonconcur.  We  reiterate  our  belief  that  the  CNA  field  progiam 
must  be  examined  from  a  quafitative  perspective  and  not  simply  a  quantitative  one, 
i.e.,  salary  and  benefit  costs  of  individual  analysts  in  the  field.  As  we  have  stated 
in  our  reqionse  to  Recommendation  3,  the  position  die  auditors  have  taken 


TABO 


Final  Report 


Revised 
P.  5 


Revised  to 
$6.2  million 
P.  5 


175 


DEPARTMENT  OF  THE  NAVY  COMMENTS  (cont'd) 


ftgafding  CNA’»  field  prognun  illustrates  a  (undainental  lade  of  understanding  of 
die  unique  nature  of  FFRDCs.  They  have  fiJled  to  recognize  that  FFRDCs  roust 
be  viewed  in  the  aggregate  when  assessing  their  long-term  special  competencies 
and  capabilities.  The  value  of  the  CNA  field  program  must  judged  as  a  v,*ole,  not 
by  limiting  the  examination  to  the  superficial  notion  of  cost  of  a  given  individual 
aiulyst.  See  our  comments  to  Recommendation  3  regarding  the  benefits  the  DON 
receives  from  the  CNA  field  program. 


TTie  Kvkwt  and  jwtiBcatioof  for  the  10  FFRDCs  did  not  «deqmidy  document  the  5  ircu  nquind  by  dte  FAR 
lad  OFPP  Policy  Letter  and  cootained  unproven  ttatemeott  to  justify  ^  aole-aourco  cootiacfc  The 

leviewt  and  jutlificatioM  did  not  raffieieotly  document  the  qwoson’  miisioa  requirements  to  be  performed  by 
tbe  FFRDCs,  the  ilceraitivcs  considered  to  inert  sponsor  requiremenU,  the  efTicieocy  and  efTectivcoeas  of  Ibe 
FFRDC  in  mertinf  sponsor  needs,  the  management  controls  established  by  the  FFRDCs  to  ensure  cost-effective 
operations,  and  (be  determination  that  criteria  for  establidung  the  FFRDC  were  still  valid  and  that  an  adequate 
sponsoring  sgieemeol  existed. 

nON  Position:  Nonconcur.  See  response  to  Recommendation  la  (Tab  B)  and 
Findings  1 ,  5,  6, 1 1 ,  and  13  CTab  D  above  and  below). 


Two  eomprehensive  reviews  were  not  approved  until  after  the  cooiiactt  wwe  awarded. 

DON  Position!  Partially  concur.  We  acknowledge  flat  the  memorandum 
promulgating  the  results  of  the  comprehensive  review  of  CNA  ^  signed  after  flw 
contract  was  awarded;  however,  as  we  sated  in  our  commena  in  Tab  C,  all^ 
parties  to  the  contract  renewal  process  were  aware  of  flie  contena  of  flhe  levfcw 
prior  to  flte  contract  award. 


Revised 
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PARAGRAPH); 

The  documeots  for  compidieiisive  reviews  did  not  detail  future  requiremeots  for  any  of  the  10  FFRDCs  in 
lenm  of  pacifically  who  needed  what  servicaa,  where,  when  and  how. 

TMTN  Position;  Nonconcur.  Once  again,  the  auditors  have  displayed  a 
fundamental  lack  of  understanding  of  the  nature  of  FFRDCs.  FFRDCs  exist  to 
provide  an  available  ongoing  resource  of  research  and  development  of  flie  major 
subsiandve  issues  facing  the  sponsoring  agency.  CNA  is  often  called  upon  to 
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examine  issues  which  quinol  be  rigidly  or  explicidy  defined  (as  is  the  case  of  work 
assigned  to  in-house  or  other  contractor  resources).  Indeed,  an  issue  may.  be 
evolving  in  scope  and  CNA  is  utilized  because  their  analysts  are  particularly 
skilled  at  assisting  the  DON  in  determining  the  breadth  of  the  'problem.' 
Additionally,  there  are  ksues  which  may  emerge  suddenly  and  require  the  quick 
response  of  the  FFRDC.  In  neither  of  these  instances  could  the  requirement  be 
defined  to  the  extent  the  auditors  appear  to  require. 

;  The  CNA  research  program  is  developed  one  year  at  a  time.  To  overspeci^  the 
research'  requirements  (p^cularly  years  in  advance',  for  such  areas  as  global 
,  regioi^  instabili^,  advancing  technologies,  and  strategic  restructuring)  is  heither 
:  advantageous  or  desirable.  The  fact  that  the  FAR  and  OFPP  Policy  Letter 
recognize  the  need  for  a  long-term  relationship  between  the  sponsor  and  the 
FFRDC  in  order  to  encourage  and  maintain,  among  other  things,  quick  response 
capability,  lends  credence  to  the  argument  that  a  determination  of  specific 
requirements  far  in  advance  is  not  required.  Renewal  of  an  FFRDC  contract  is 
based  upon  a  continuing  research  and  development  need  of  the  sponsoring  agency, 
not  upon  specific  requirements. 

FINDING  6:  ALTERNATIVE  C:ONTRACTOR  AND  GOVERNMENT  SOURCES 
(PG  9  OF  THE  REPORT.  2ND  PARAGRAPH); 

Sutements  ia  the  review  documents  and  sole-source  justifications  that  no  otter  sources  existed  were  arhitrarjr 
and  unsupported.  Formal  mailcet  surveys  were  not  performed  for  any  of  the  10  FFXDCs  to  determine  wbetter 
private-sector  firms,  id>bouse  Oovemment  personnel,  or  otter  FFRDCs  could  perform  the  work  even  ttoo|k 
users  stated  that  alterealives  to  tbe  FFRDCs  did  exist. 

DON  Position;  Nonconcur.  FAR  35.017-4(cKl)  states  that  a  comprehensive 
review  should  include  "[clonsideration  of  alternative  sources  to  meet  tte  sponsors 
needs.*  There  is  no  requirement  that  market  surveys  must  be  conducted.  CNA 
has  provided  the  DON  wth  more  than  50  years  of  independent,  unbiased, 

.  audioritative,  analytically  rigorous  research.  It  has  done  so  dirough  its  utilization 
of  its  unique  hands-on  familiarity  with  fleet  operations  and  access  to  sensitive  data. 
Thus,  in  considering  alternative  sources  that  could  possibly  provide  the  same  level 
of  research  to  the  DON,  the  qualitative  factors  surrounding  a  continuum  of  SO 
years  of  analytical  support  were  given  substantial  weight.  A  formal  market  survey 
would  not  have  been  an  effective  tool  in  this  regard.  Additionally,  per  die  FAR,  it 
is  the  responsibility  of  the  sponsor  to  determine  the  means  by  which  reviews 
(including  alternative  imurce  consideration)  are  to  be  conducted. 
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Deleted 


pp.  8  and  9 


Nooe  of  the  reviewi  or  tote-source  justincatioos  for  the  10  FFRDCt  furtually  established  that  only  one  source 
could  effectively  accomplish  the  ^asors*  requirements.  Accordingly  we  believe  that  the  DFARS  should  be 
expanded  to  require  advance  notice  before  aitd  FFRDC  contract  is  renewed.  An  alternative  it  for  the  Director 
of  Defense  Procurement  to  propose  a  revision  to  FAR  5.202(aKI0)  and  5.205(b).  Appendix  Q  abows  language 
can  be  used  to  revise  the  acquisition  regulations.  During  the  interim,  DDRdS:E  ^ould  require  DOD 
spoosors  to  publish  in  the  CBD  advance  notices  of  interest  that  identify  procureraent  requiiements. 

PON  PosHion:  Nonconcur.  See  response  to  Recommendations  Ic  and  4  (Tab  B). 


Hone  of  the  leviewi  or  aole-source  justifications  for  the  10  FFRDCs  cited  low  Federal  ularies  for  acieotific  and 
toriinirat  positioos  as  t  ftasoo  wfay  the  Federal  Government  could  not  attract  the  scientific  and  technical 
penonnet  to  perform  required  tasks  in-house. 

PON  Position:  Nonconcur.  While  we  acknowledge  that  Federal  salary  levels 
were  not  specifically  mentioned  in  the  comprehensive  review,  the  fact  that  tte 
FAR  supports  a  long-term  relationship  with  the  FFRDC  in  order  to  attract  hithi 
qunWtyf  personnel  to  the  FFRDC  implies  that  the  ability  to  attract  such  personnel  to 
the  Government  with  a  degree  of  continuity  is  unlikely.  It  has  also  been  widely 
reported  over  the  last  several  years  that  Federal  salary  levels  (particularly  military 
salaries)  lag  considerably  behind  the  private  sector.  Experienced  Government 
officials  are  well  aware  of  the  difficulties  the  Government  faces  in  recruiting  and 
retaining  substantial  numbers  of  individuals  with  post-masters  technical  degrees. 


ladepesdefice  tnd  objectivity  were  cited  u  general  reasons  why  piivaie-sector  finns  could  not  perfora  foe  tasks 
of  foe  FFRDCs.  However,  foe  reviews  and  justifications  did  not  address  why  other  nonprofit  organizations, 
including  tome  former  DOD  FFRDCs.  would  lack  indepeodeoce  and  ohjectivity. 

PON  Position:  Nonconcur.  Once  again  the  auditors  appear  to  have  missed  the 
point  regarding  the  independence  and  objectivity  of  an  FFRDC.  Independence  and 
dbjeedvity  are  salient  factors  in  comparing  an  FFRDC  to  private-sector  firms  and 
in-house  resources  as  well.  When  comparing  an  FFRDC  to  other  nonprofit  firms, 
the  overriding  area  of  concern  is  not  independence  and  objectivity  (although  they 
are  certainly  considered);  it  is  the  special  competencies  and  capabilities  of  the 
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FFRDC  which  become  paramount.  In  the  case  of  CNA  Aese  would  include  their 
50  year  continuum  of  expertise  in  naval  matters;  hands-on  exposure  to,  ^ 
fiuiuliarity  wiA,  fleet  operations;  access  to  sensitive  data;  and  quick  response 
capability. 


Final  Report 


We  leviewed  dte  epproprieteoes*  of  Ihe  DOD  sponsors'  methods  for  evsiualinf  FFRDC  efficiency  end 
eQecUyencss  in  SK^ins  user  needs.  We  also  reviewed  ilv!  reliaViljpf  of  Ihe  dau  repotted  in  user  assessments 
(1^  tummariied  in  the  comprehensive  reviews  by  Ibe  priawy  DOD  sponsofs.  We  coocluded  diat  user 
rrqwH'—  and  primary  DOD  sponsor  IsseiHUeots  of  ilK  i^FRDCt'  efficiency  Siod  cffediveaess  in  Ibe 
ooo^tdieasive  reviews  amounted  to  little  mote  than  unsubstantiated  penooat  opinion. 

DON  Position!  Nonconcur.  The  method  used  by  Ac  DON  for  evaluating  *e 
effidency  and  effectiveness  of  CNA  was  to  obtain  feedback  from  CNA’s 
'customers.'  To  collect  data  regarding  CNA  in  this  manner  is  entirely 
appropriate;  customer  feedback  surveys  are  a.  widely  accepted  meAod  of 
i  determining  the  quality  and  utility  of  products  produced.  In  our  t^inion,  the 
concept  of  employing  a  sponsor  questionnaire  reflects  the  spirit  and  meAodOlogy 
of  Total  Quality  Leadership.  Who,  other  Aan  Ae  user,  is  better  qualified  to 
comment  upon  the  quality  and  utiliQr  of  the  research  producu  produced  by  CNA? 
FurAer,  the  FAR  is  silent  on  a  specific  method  for  conducting  Ac  com|Mehensive 
review,  and  Aus,  determination  of  a  method  tests  wiA  Ae  sponsor. 


Revised 
P.  9 


Prognm  officialt  who  prepared  Ihe  eummeiy  oompreheimve  review  documents  did  uot  adequatdy  doeumeal 
foeirasseetmeiilormui(eiiienlcoairoUforwiy  oftfaeDODFFRDO.  Ibe  review  documeau  tenenlly  slitad 
Am  D(M>  edimiiistntive  comracting  officers  (ACO).  the  Defense  Cootracl  Audit  Agency  (DCAA).  and  DOD 

poli^  wd  advisoty  graupc  that  provided  overeighi  to  each  FFRDC  ensured  operalroos  were  coet-eRacUva. 

Nooeoffoeievicwy  documcaled  the  inlcraal  FFRDCh  took  to  ensure  Ihti  aecinale  aecoualiitg  lecoidt 
were  maiataiaed,  that  sound  procuremeal  practices  were  followed,  that  ovetbead  costs  were  teesouahle.  or  foal 
aMaageaseai  foes  were  for  ctseatial  opeteting  lequiiemeatt. 

DON  Position;  Nonconcur.  There  is  no  legulatoiy  provision  that  prohibits 
sponsors  from  relying  upon  Ac  advice,  expertise,  luid  conclusions  reached  by 
othn'  Government  organizations  in  conducting  comprehensive  reviews.  It  is 
proper  to  rely  on  Ihe  assurances  of  entities  such  as  Ae  ACO  and  DCAA,  because 
it  is  within  Ae  mission  and  function  of  Aese  agencies  to  provide  contract 
oversight.  ACOs  are  responsible  for  conducting  procurement  systems  reviews, 
property  management  systems  reviews,  and  reviewing  and  approving  proposed 
subcontracts.  The  DCAA  conducts  proposal  audits,  incurred  cost  audits,  and 
reviews  of  accounting  systems  and  proc^ures.  To  require  primary  sponsoring 
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P.  10 


Kviews  of  accounting  systems  and  procedures.  To  require  primary  sponsoring 
offices  to  conduct  and  document  these  reviews  and  audits  separately  would  be 
redundant  or  excesrive. 

The  term  'management  fee*  as  it  currently  applies  to  CNA  is  Synonymous  with 
fixed  fee  from  which  'retained  earnings”  or  'reserves*  as  used  in  FAR 
3S.0l7-l(c)(3)  are  derived.  Since  the  FAR  requires  that  FFRDC  sponsoring 
agreements  identify  retained  earnings,  and  that  a  plan  for  their  use  be  developed,  it 
can  be  concluded  diat  the  FAR  envisions  such  retained  earnings  as  reasonable  and 
appropriate.  CNA  as  a  not-for-profit  organization  may  receive  a  fee  as  long  as  it 
does  not  accrue  to  die  benefit  of  an  individual  or  stodc  holders.  Though  the  FAR 
requires  that  an  FFRDC  submit  to  the  ^nsoring  agency  a  plan  for  the  use  of  its 
retained  earnings,  die  FAR  does  not  require  that  management  fees  be  used  for 
'essential  operating  requirements.* 

FINDING  12t  COMPLIANCE  WITH  FFRDC  CRITERIA  (PG  12  OF  THE 

REPORT.  1ST  FULL  PARAGRAPH); 


Hke  review  documeats  did  not  specifically  address  the  sponsoring  agreemeats. 

DON  Position;  Concur.  A  specific  statement  regarding  compliance  of  the 
sponsoring  agreement  with  the  requirements  of  FAR  35.017-1  was  not  included  in 
the  review  documents.  A  statement  to  this  efTect  will  be  included  in  fiiture 
comprehensive  reviews. 


FINDING  13:  COMPXlANCELWrm  FFKOC  CRITERIA:  ESTABLISHMENT 
CRITERU  fPC  12  OF  THE  REPORT,  2NP  FULL  PARAGRAFffi: 


As  fnquiftd  by  FAR  35.017-2  none  of  the  review  documeoU  for  the  10  FFROCi  adequately  established  that: 

•  the  mtssioo  of  the  FFRDC  was  sUted  cleariy  eoough  to  differentiate  between  work  that  should  be 
pcffomed  by  die  FFRDC  and  work  that  d»ould  be  peifonned  by  md-FFRDCs. 

•  conventual  contracting  melhoda  and  existing  in-bouae  fidlitiea  could  not  tffeetively  aatisiy  the 
aponsor’a  special  RAD  requiiemcnls. 

-  cootsoU  wete  in  place  to  ensure  the  cost  of  FFRDC  aervioes  were  leasooihle. 


DON  Position!  Partially  concur.  We  acknowledge  that  the  comprehensive  review 
documents  do  not  include  an  explicit,  specific  statement  that  provides  a 
determination  that  the  criteria  for  establishing  the  FFRDC  had  continued  to  be 
satisfied.  However,  the  documents  do  address  close,  proactive  oversight  by  Navy 
officials  to  ensure  compliance  vrith  OFPP  Policy  Letter  84-1.  Future 
comprehensive  reviews  will  include  specific  statements  addressing  whether  or  not 
the  FFRDC  continues  to  comply  with  FFRDC  establishment  criteria. 
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[Finding  14:  Support  for  Justifications  (Pg.  13  of  the  Report, 

1st  Full  Paragraph] 

The  writteo  reviews  did  not  support  die  M«*<ouret  justifiMtiocis.  Tedmicil  persoaoel.  oontiKtini  ofRcen. 
senior  aoquisitioo  official  certified  to  the  accuracy  and  completeaess  of  each  sole-source  justification  allbouih 
ooD^rtbeasive  reviews  for  9  of  the  10  FFRDCS  were  not  done  at  the  time  sole-source  justificaiiooi  were 

pfqwred.  None  of  the  seven  reviews  that  were  dooe  addressed  all  five  crilciia  from  the  OFFP  and 
ftttdaoce  for  letaioini  ad  FFRDC,  The  Under  Secretary  of  Defease  for  Acquisition,  the  Service  Acquisition 
Executives,  die  DpR&E,  and  the  cootracliaf  officers  should  have  dudleaH  ^  Kcuncy  and  cx^leleoess  of 
the  dctenninations  in  the  sole-source  justifications  and  should  have  delennined  vdiether  the  assatioos  in  each 
justification  were  supported  by  a  comprehensive  review  document.  The  justifications  did  not  conUin  sufficient 
frets  and  rationale  to  justify  the  use  of  FAR  iS.302-3  for  sole-source  swards  to  the  10  FFRDCs* 

DON  Position!  Nonconcur,  This  app^  to  repeat  many  of  the  previous 

findings  and  siatements  of  the  report  and  is  reiti^fated  by  Recpmihendations  Ic  and 
4,  Therefore  our  position  regarding  this  finding  can  be  found  in  our  response  to 
Recommendations  tc  and  4  (Tab  B),  Tab  C,  and  in  our  comments  concerning 
Findings  1  and  4  (Tab  D  above). 

FINDING  15;  SUPPORT  FOR  JUSTOTCATIONS  (PG  13  OF  THE  REPORT.  3NP 
FULL  PARAGRAPH); 

Wc  found  no  evideace  that  the  contracting  officers  of  senior  acquisition  officials  ever  requested  additional 
dpeumeotatioD  or  rationale  for  the  sole-source  contracts  before  approving  iht  sole-source  justifications. 

DON  Position:  Nonconcur.  Sec  our  comments  in  Tab  C  and  those  concerning 
Finding  4  (Tab  D  above). 

FINDING  ISrFUTURE  AWARDS  (PG  13  OF  THE  REPORT.  LAST 
PARAGRAPH); 

n.  MiliUiy  DqwrtaieaU  or  tbe  Defease  Supply  Service-WisUaftoo  (DSS-W)  diouldl  aa(  a««rd  a  S-yeer 
wle-Mone  ooabMt  to  tay  FFRDC  or  exercise  lay  options  or  modiSesiioo  to  an  cxitfiag  cootruct  with  an 

FFKDC  imtil  aa  a4<iquata  coaiprebeasive  review  aad  lole-sourcajutlificatioa  it  completad.  FFRDC  latviect 

ttat  can  not  te  properly  justified  should  be  discoolioued.  DOD  tfcmon  should  conpeie  to  die  msxinwin 
cxtcal  po»l>le  missaoD  requitemeals  or  technical  needs  tequiiini  FFRDC  perfoiBMaee. 

DON  Poslfton;  Nonconcur.  See  response  lo  Recommendations  Id  and  2  (Tab  B). 

The  FAR  and  Titles  10  and  41  of  the  U.S.  Code  veiy  clearly  provide  authority  lo 
exempt  FFRDGs  from  the  requirement  to  compete  contract  renewals,  once  it  has 
been  determined  the  Government  continues  to  have  need  of  the  services  of  the 
FFRDC.  Such  statutory  and  regulatory  provisions  aflirm  evidence  that  to  establish 
and  maintain  a  long-term,  suble  relationship  between  an  FFRDC  and  the 
i  Government  is  a  fundamental  policy  objective.  The  finding  ignores  this  and  such 
ign(»an<»  is  indicative  of  a  failure  to  acknowledge  that  the  Government  is  well 
served  in  its  ongoing  use  of  FFRDCs,  and  that  the  current  austere  budgetary 
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ignorance  is  indicative  of  a  failure  to  acknowledge  that  the  Government  is  well 
served  in  its  ongoing  use  of  FFRDCs,  and  that  the  current  austere  budgetary 
climate  necessitates  an  even  greater  reliance  on  FFRDC  capabilities  in  helping  to 
shape  the  future  of  the  nation’s  armed  forces. 

FINDING  17;  EFreCT  OF  SOLE-SOURCE  CPhTTRACTS;  CNA  FIELD 

PROGRAM  a»G  14  OF  THE  REPORT.  2ND  PARAGRAPHI; 

In  collecting  informstioD  on  CNA  field  loelysttp  we  estimsted  that  during  FYs  1994  tod  1995»  tlie  Navy  could 

eliminate  about  $7.8  million  of  costa  by  converting  the  41  CNA  field  analyste  to  Coveminefit  GM-15  analysts. 

DON  Position:  Nonconcur.  See  our  response  to  Recommendatioii  3  (Tab  B)  and 
our  comments  on  Finding  2  (Tab  D  above).  In  addition  to  our  previous 
comments,  it  should  be  noted  that  an  actual  cost  comparison  has  not  been 
accomplished  for  two  reasons:  primarily,  we  directly  oppose  the  premise  of  the 
auditors  that  retention  of  the  CNA  Field  prognun  is  solely  a  matter  of  cost; 
secondarily,  the  report  does  not  provide  the  baris,  i.e.,  assumptions,  used  to  derive 
the  dollar  amounts  reported.  It  is  also  true  for  the  table  displayed  in  Appendix  J. 

FINDING  18;  EFFECT  OF  SOLE-SOURCE  CONTRACTS;  IN-HOUSE  VS. 

CONTRACTOR  PERFORMANCE  COSTS  (PG  14  OF  IHE  REPORT.  3RD 

PARAgyAMf): 

We  ate  unawaie  of  ui-bOttse-veRus<ontFietor  perfornumoe  cost  comparisons  for  any  of  the  coatfaded  advisory 

and  aasisUDce  services  provided  by  any  of  the  10  FFRDCs. 

DON  Position!  Nonconcur.  Here  again,  the  auditors  appear  to  proceed  from  a 
purely  cost  related  perspective.  We  maintain  that  FFRDCs  must  be  judged  upon 
their  qualitative  capability  to  meet  some  special  long*term  research  and 
development  need  of  the  sponsoring  agency,  not  purely  quantitatively,  i.e., 
monetarily.  The  special  capabilities  and  competencies  of  OOD  FFRDCs  have,  in 
almost  all  cases,  tmn  acquired  and  enhaiKed  over  a  substantial  period  of  time 
(more  then  SO  years  in  ll^  case  of  CNA).  FFRDCs  were  originally  created  in 
order  to  assemble  a  pool  of  highly  educated  and  talented  iiKlividuals  to  provide  an 
iiKlependcnt,  long-term,  and  dependable  body  of  scientific  knowledge  otherwise 
unavailable  within  the  Government  or  private  sector.  It  remains  true  today  that  the 
Government  has  need  of  a  concentrated  supply  of  research  and  development 
capabilities.  FFRDCs  serve  this  purpose,  aiKl  should  be  recognized  for  tfie 
valuable  national  asset  they  are. 
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FINDING  19;  POP  FFRDC  MANAGEMENT  PLAN  (PC  IS  OF  THE  REPORT. 

At  of  Januuy  14,  1993,  tlw  comprehcAsive  reviews  and  the  tpoosor  diie^vet  inquired  by  Ibe  manageineat  plan 
were  not  lubmiited  io  DDR&E  for  review. 

PON  Position:  Nonconcur.  In  the  case  of  CNA  this  finding  is  untrue.  CNA’s 
comprehensive  review  was  first  provided  to  DDR&E  at  the  time  of  its 
promulgation;  another  copy  was  forwarded  to  DDR&E  in  January  of  1992. 
Although  spine  sponspr  directives  relating  to  CNA  were  informally  provided  to 
DDR&E  at  various  times  Ihroughpiit  19Jf2, ,  aH  pertinent  directives  were  forwarded 
by  formal  memorandum  oil  8  January  WS. 
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DODIG  Draft  Audit  Report  of  Mardi  23. 1993 
on 

Sole-source  Justification  for  DOD-sponsored 
Federaliy  Funded  Research  and  Development  Centers 
Project  No.  ICH-5012.02 
(Appendix  J) 


P.  76 


Hie  review  document  wss  not  prepired  and  approved  until  November  30,  1990,  after  the  sole-source 
justification  approval  and  contract  award  on  ^tember  28,  1990. 

DON  Position:  Nonconcur.  See  comments  in  Tab  C  and  those  on  Rnding  4  fTeh 


PP.  76  and  77 


txiij  AV.x  I  wauat  > ;  i  ai;t  ai’i  i  AOPailt  iiKa  « iji  Aitiiffl  ;t  u  lu  it; 


Deleted 


The  review  document  did  not  adequately  define  future  requirements  and  did  not  address  tbe  cdst-eflectivaiets  of 
CNA  services  or  why  the  services  could  not  be  provided  by  a  combination  of  additional  Navy  personnel  and 
private  conbaetors. 

DON  Position;  Nonconcur.  ^  response  to  Recommendation  la  (Tab  B)  and 
comments  on  Findings  S,  6,  II,  and  18  (Tab  D). 


Tbe  requirement  were  not  qmopsized  in  the  CBD. 

DON  Position:  Nonconcur.  See  response  to  Recommendations  Ic  and  4  (Tab  B). 


P.  77 


Tbe  review  document  also  did  not  provide  an  adequate  rationale  for  transferring  tbe  operation  of  CNA  from  tbe 
Hudson  Ifutiniie  to  CNA,  Incorporated,  a  new  non-profit  coiporalion.  Neither  document 
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eonuiof  ■  fonnd  Mplamiioii  of  why  the  Nivy  did  not  coiq^iively  wiicil  prapotib  fcr  •  tew  opcniar. 

DON  Position:  Nonconcur.  This  is  a  superfluous  issue.  FAR  S3.017r2(h)  stales 
that  an  FFRDC  may  be  '...operated,  managed,  or  administered,  by  an  autonomous 
organizadon...”  Thus,  the  FAR  envisions  organizadonal  structures  such  as 
CNA's.  If  the  sponsoring  agency  determines  that  an  FFRDC,  as  an  autonomous 
corporate  endty,  'meets  some  special  long-term  research  or  development  need 
which  cannot  be  met  as  effectively  by  existing  in-house  or  contractor  resources* 
[FAR  35 .01 7(a)(2)],  it  would  be  neither  reasonable  nor  necessary  to  pursue  some 
other  outside  source  of  ipanagement  for  dial  FFRDC.  Once  the  DON  agreed  witii 
the  detennihalion  tharCHA  had  reached  a  point  of  effective  self-maitiigement, 
there  was  no  logical  in^sbh  for  the  Navy  to  deliberately  impose  anotiiCr  parent 
corporation  to  manage  CNA.  Such  an  action  would  have  only  resulted  in  inserting 
an  unnecessary  layer  of  management  and  cost.  An  FFRDC's  stahis  as  a  private 
corporation  is  not,  in  and  of  itself,  improper  or  illegal  and  is,  in  fact,  typical  of 
American  business  entities. 


Audit  response 
P.  79 


i  al  a  a  (al  i  atff  at  ti  a  ajLtiaiilllg)  Hejill  ll 


P.  77 


CNA’s  operatioMl  efTicicncy  «nd  effectiveaess  was  assessed  on  die  basis  of  subjective  ratiass  of  wbeiber  CNA 
met  the  analytic  needs  of  the  customer  and  was  timely,  and  did  not  discuss  tbe  relisbility  of  its  dsta  oollectioa 
and  analytis.  r^ftint.  computer  opeialioos,  and  orfanixatioaal  structure. 

DON  Position!  Nonconcur.  See  comments  on  Findings  10  and  1 1  (Tab  D). 


Tbe  review  document  also  did  not  document  the  mana|ement  controls  implemented  by  CNA  to  assure  enst- 
effective  opeietioo  and  did  tiol  define  die  mission  of  CNA  ckaily  enou^  .lo  difierentiste  CNA  wMk  Iraaa  woik 
appropriate  for  in-bouse  or  non-FFRDC  cootractors. 

DON  Pacition!  Nonconcur.  See  comments  on  Findings  10  and  1 1  (Tab  D).  It  is 
true  that  one  of  the  criteria  for  establishing  an  FFRDC  is  that  the  sponsor  shall 
ensure  'The  basic  purpose  and  mission  of  the  FFRDC  is  staled  clearly  enough  to 
enable  differentiation  between  work  which  should  be  performed  by  die  FFRDC 
and  that  which  should  be  performed  by  non-FFRDCs.”  However,  diere  is 
ABSOLUTELY  NO  REQUIREMENT  that  this  definition  must  be  contained  in  die 
comprehensive  review  documents.  To  foult  the  sponsor  for  not  including  such  a 
definition  is  entirely  unsupported. 
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TSSTTE  li  EVALUATION  OF  THE  REVIEW  AND  JDSTinCATION;  FAB 

CRITERU  FOR  ESTABLISHING  AN  FFRPC  AND  MAINTAINING.  A 

SPONSORING  AGREEMENT  fPG  71  OF  THE  REPORT.  1ST  FULL 

PARAGRAPH!; 

The  f«view  documeot  also  did  noc  adequately  address  compliance  with  the  FAR  criteria  for  establishinf  an 
FFRDC  Of  for  maiataminf  a  sponsoring  agreement* 

PQN  Position!  Partially  concur.  See  comments  on  Findings  12  and  13  (Tab  D). 

TSSITR  Xt  EVALUATION  OF  THE  REVIEW  AND  JUSTOICATION;  REVIEW 

nOTIlMENT  STATISTICS  fl>G  71  OF  THE  REPORT.  2ND  FULL  PARAGRAPH); 

We  (Iso  deteraaioed  Ibd  the  lUlisItcs  in  the  leview  document  resardinx  user  consensus  about  CNA  manaxement 
demonstnlinx  a  cost-cflectivc  operation  were  misleadinx-  Navy  and  Marine  Coipa  activities  reqwndinx  to  the 
survey  were  not  in  a  poritioo  to  comment  on  tbe  cost-eflectivenesa  of  CNA's  opentioo  because  the  Navy 
ccalnlly  funded  CNA  studies  and  individual  users  were  not  provided  an  sccountinx  of  costs  for  their  pnyects. 

PQN  Position;  Partially  concur.  We  acknowledge  that  Navy  and  Marine  Corps 
activities  did  not  have  individual  project  cost  data  available  to  diem  at  the  time  of 
the  comprehensive  review  survey.  However,  the  documents  do  address  close, 
proactive  oversight  by  Navy  officials  to  ensure  compliance  with  OFPP  Policy 
Letter  84-1.  The  CNA  Policy  Council  (whose  members  include  the  most  senior  of 
don  civilian  and  uniformed  leadership)  were  periodically  briefed  regarding  funds 
for  the  CNA  research  program.  The  Policy  Council  has,  as  one  of  its 
responsibilities,  made  decisions  regarding  binding  levels  for  CNA.  Based  on  their 
actions,  and  their  opinion  of  the  caliber  of  research  conducted  by  CNA,  they 
remain  satisfied  that  CNA  provides  a  valuable  service  to  the  DON  at  a  reasonable 
cost. 

In  addition  to  the  oversight  responsibilities  of  the  CNA  Policy  Council,  a  newly 
adopted  management  process  for  CNA  places  greater  emphasis  on  centralized 
oversight,  proactive  role  of  senior  DON  leadership  (3-star/ASN)  and  CNA 
leadership,  and  cradle  to-grave  tracking  of  individuid  CNA  projects  including 
costs).  As  a  result  of  this  new  management  process,  users  will  have  individual 
project  cost  data  available  to  them  during  future  comprehensive  reviews. 
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ISSUE  9;  COST-EFFECTIVENESS  OF  CNA  FIELD  ANALYSTS  (PG  71  OF  Iffi 
REPORT.  LAST  PARAGRAPH.  CONTINUEP  ON  PgJg}; 


Over  lbe  2  mnuniai  yew*  of  ibe  CNA  eontr»cl  (FY»  1994  tod  1995),  *boui  $7.$  millioo  of  cod  (2  y««*  d 
$3.9  million  per  yeer)  Can  be  cUmineteil  by  using  in-house  civiliin  persoonel  instetd  of  CNA  field  analysis. 

DON  Position;  Niinconcur.  See  response  to  Recommendation  3  CTab  B)  and 
comments  on  Findings  2  and  17  (Tab  D). 


Although  they  are  assigned  to  various  Navy  and  Marine  Corps  commands  and 
vrti>oit  diittUy  to  their  respective' commanders,  CNA  analysts  are  prohibited  from 
-  performing  routine  staff  arid  admijustrative;  QBNAV  Insmictioo  and 

Marine  Corps  Order.  CNA  field  representatives  me  tasked  with  providing 
analytical  support  concerning  issues  of  interest  to  the  commander.  The  CNA  Vice 
President  for  Field  Operations  is  responsible  for  evaluating  the  qua%  and 
effectiveness  of  the  field  representatives’  performance  based  not  only  on  input 
from  the  commander,  but  also  on  a  CNA  review  of  Ihe  field  representatives’ 
analytical  products. 

ITie  fuil  scope  of  field  analysts’  activities  encompasses  such  issues  as  base 
consolidation,  joint  operations  adaptation,  and  fleet  concepts  of  operation.  They 
also  provide  independent,  immediate,  and/or  quick  response  analytical  support  to 
actual  operations,  as  in  Operations  Desert  Shield/Storm  and  Restore  Hope,  as  well 
as  post-operational  analysis  of  "fessons  leamcd"  for  operations,  systems, 
lequiremente,  support  activities,  and  exercises. 

Such  activities  are  inconsistent  with  the  report’s  assertion  that  "CNA  used  their 
field  program  as  a  training  ground  for  junior  analysts.*  Some  field  representa¬ 
tives,  inevitably,  are  serving  in  their  first  field  assignment,  after  having  worked  at 
CNA  for  at  least  two  years,  but  often  longer.  Many  field  representatives  have 
served  several  lours.  All  CNA  analysts  assigned  as  field  representatives  bring  to 
the  commands  to  which  they  are  assigned  a  rigorous  academic  background, 
training,  experience,  and  the  full  range  of  CNA  support. 

It  continues  to  remain  a  serious  and  fundamental  flaw  of  this  report  that  it  fails  to 
comprehend  die  value  of  the  CNA. field  program. 
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DEPARTMENT  OF  THE  AIR  FORCE  COi^ENTS 


DEPARTMENT  OF  THE  AIR  FORCE 

WASHINGTON  DC  aOMO'IOOO 

]m  i?J3 

orricc  Of  thc  tccncTAnv 


MEMORANDUM  FOR  ASSISTANT  INSPECTOR  GENERAL  FOR  AUDIT 
FOLLOW  UP,  OFFICE  OF  THE  INSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 

SUBJECT:  Air  F6ri^  Resi^^  to  DoD  IG  Draft  Audit  Report  on  Sdle^Sotirce 
Justification  for  OoD-Sponsored  FederaUy  Funded  Research  and 
Development  Centers  (Project  lCH-50 12.02) 

-  INFORMATION  MEMORANDUM 

This  rbemorandum  is  in  reply  to  your  request  for  comments  on  subject  report. 
The  draft  audit  does  not  take  into  consideration  the  positions  stated  in  our  response 
of  March  23, 1993,  to  the  ''working*'  draft  audit  All  of  our  previous  comments  are 
still  valid  and  should  be  considered  in  the  final  report  in  addition  to  those  contained 
below. 

Recommendation  1  •  Partially  concur.  This  recommendation  calls  for  new 
comprehensive  reviews  of  all  DOD-sponsored  FFRDCs.  We  concur  that  new 
comprehensive  reviews  are  warrant^.  These  reviews  are  being  conducted,  or  will 
soon  be  conducted,  at  all  Air  Forc^sponsored  FFRDCs.  While  concurring  with  the 
necessity  of  new  reviews,  we  have  concerns  about  the  ground  rules  associated  with 
the  conduct  of  the  reviews.  It  is  essential  that  comprehensive  reviews  be  conducted 
with  the  understanding  that  FFRDCs  are  a  unique  type  of  national  resource  that 
are  deliberately  protected  from  normal  competitive  pressures.  FAR  6.302-3  states: 

"Full  and  open  competition  need  not  be  provided  for  when  it  is 
necessary  to  award  the  contract  to  a  particular  source  or  sources  in 
order  to...e8tabli8h  or  maintain  an  essential  engineering,  research,  or 
development  capability  to  be  provided  by  an  educational  or  other 
nonprofit  institution  or  federally  funded  research  and  development 
center." 

The  legislative  history  of  CICA  makes  it  clear  that  Congress  recognized  the 
unique  and  protected  nature  of  FFRDCs  as  evidenced  by  the  following  quote  from 
Senate  Report  #98-297: 
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Eteconraendation  l.c. 
P,  18 


"As  the  committee  noted  above,  these  insdtutiozui  make  a  vital, 
continuing  contribution  to  our  national  defense.  They  cannot  be 
toUUy  exempt  from  the  rigors  of  competition.  They  are  not  today. 

Still  the  Committee  feels  that  among  paramount  concerns  in  awarding 
research  contracts  should  be  the  qualifications,  resources  and 
expertise  of  the  institution  or  organization.  Iherefore,  the  Committee 
felt  it  was  imperative  to  provide  agency  heads  with  a  degree  of 
flexibility  to  establish  or  maintain  an  essential  research  capability, 
without  using  competitive  procedures,  to  be  provided  by  specific  types 
of  institutions  or  organizations." 

It  is  also  essential  that  comprehensive  reviews  be  conducted  with  the 
understanding  that  FFRDCs  support  the  military  departments  in  dealing  with  the 
most  complex  technical  problems  that  relate  to  the  most  critical  aspects  of  national 
security  policy.  The  statements  of  work  associated  with  this  type  of  support  may 
necessarily  be  broad  in  scope.  Providing  on*going  support  to  the  government  on  the 
most  complex  issues  relating  to  national  security  is  ftodamentally  different  firom 
the  type  of  support  provided  by  typical  service  contractors. 

We  believe  that  comprehensive  reviews  should  be  conducted  using  the 
guidelines  oonUined  in  FAR  35.017-4(c)  rather  than  the  review  guidelines 
contained  in  Recommendation  l.b.  of  the  audit  report  The  guidelines  in  the  audit 
report  are  not  the  same  aa  those  in  the  FAR.  The  FAR  guidelines  represent  a  better 
balance  of  the  necessity  to  protect  the  governments  interests  while  recognizing  the 
unique  nature  of  FFRDCa. 

Recommendation  l.c.  of  the  audit  report  asserts  that  as  part  of  the  review 
process,  FFRDC  sponsors  should  publish  advance  notices  of  intent  in  the 
Commerce  Business  Daily  (GBD)  and  perform  market  surveys  to  validate  that 
competition  is  infeasible.  We  do  not  concur.  There  is  no  support  for  this 
recommendation  in  law  or  regulation.  The  Competition  in  ^ntracting  Act  clearly 
exempts  noncompetitive  FFRDC  acquisitions  from  the  requirement  for  pre-award 
synopsis  in  the  CBD  (See  41  UB.C.  416(cX2)).  This  statutoiy  exemption  is 
implemented  in  FAR  5.202(aK10).  The  statutory  and  regulatory  exemption  firom 
the  requirement  for  synopsis  reinforces  the  fact  that  FFRDCs  are  unique  resoureea. 

Recommendation  Id.  ia  that  the  Department  discontinue  sponsoring  any 
FFRDC  that  cannot  be  adequately  justifi^.  We  concur. 
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RidcomAidiidation  2  ?  concur.  The  recommendation  is  that  FFRDC 

•ponaora  not  award  any  oontracta  or  exerciae  any  optiona  or  modificattona  to, 
enatTng  contracU  pending  completion  of  oomprehenBive  reviews.  We  concur  with 
the  need  to  perlbrm  comprehensive  reviews  prior  to  the  award  of  new  contracts. 
However,  this  recommendation  does  not  recognize  that  options  are  ezerdsed 
annually  and  modifications  to  these  contracts,  aa  with  any  government  contracts, 
are  written  routinely.  Implementation  of  this  recommendation  would  1»  very 
diaruptive  to  the  on-going  work  of  FFRDCs  and  their  sponsors.  Such  disruption 
could  only  be  justified  if  there  was  some  reasOn  to  believe  that  a  comprehensive 
review  would  OOt  the  d>n|j|hued;0P0Osor8hip  of  FFRPQs.  There  is  po  reason 

for  such  a  belief!  Fpljlitions  aM  fobdl^tions  tofristing  contracts. 
should  be  conduct^  aia  usuid  on-goihg  compi^nensive 

reviews.  Furthermorei  if  this  recommendation  were  followed  we  Would  shut  down 
11  System  Program  Offices  (SPOs)  at  Electronic  Systems  Center  and  16  SPOs  at 
Space  and  Missile  Systeois  Center  which  would  directly  delay  on-going  program 
actions  and  contracts  collectively  valued  at  $973  million. 

Recommendatipii  d*  •  Nonconcur,  The  recommendation  is  that  the  DFARS  ^ 
amended  to  requir^  tbat  renewal  of  cohirOcU  with  DOP  FFRDCs  be  publicized  in 
advance.  We  beli^^tbat  adopting  this  recoinmehdpdn  would  tend  to  blur  the 
distinction  between  FFRDCs  and  service  contracfeihl.  Under  cux^nt  regulatory 
guidance,  the  decision  to  renew  sponsorship  of  an  FFRDC  is  not  severable  firom  the 
dedsioh  to  award  a  sole-source  contract  to  the  existing  FFRDC  orpnization.  FAR 
36.017-4  provides  only  two  options  if,  as  a  result  of  a  comprehensive  review,  a 
sponsor  deddes  not  to  continue  sponsoring  an  FFRDC.  Tlxe  sponsor  can  either  find 
another  government  agency  to  sponsor  the  FFRDC  or  phase  out  the  FFRDC. 
Recompeting  for  the  services  of  the  FFRDC  is  not  an  option.  The  FAR  coverage  is 
logical  given  the  unique  and  long  term  nature  of  an  FFRDC's  relationship  with  the 
government. 

This  recominehdation  also  is  not  consistent  i^th  OFPP  Policy  Letter  84-1 
which  states  that  "When  FFRDCs  are  esUblished,  long-term  Govenment 
relationships  are  encouraged  in  order  to  provide  the  continuity  that  will  attract 
high  quality  personnel  to  the  FFRDC.  This  relationship  should  be  of  a  type  to 
encourage  the  FRR1X7  to  maintain  currenqf  in  its  fieldfs)  of  expertise,  nwntain  its 
objectivity  and  independence,  preserve  its  fkmiliari^  with  the  needs  of  its 
aponsoris),  and  provide  a  quick  response  capabili^.”  It  also  states  that  the 
:  sponsoring  agreement  shall  contain  "A  prohibition  against  the  FFRDC  competing 
with  any  non-FFRDC  concern  in  response  to  a  Federal  agency  formal  Request  For 
Proposal  for  other  than  the  operation  of  an  FFRDC.* 

Finally,  no  regulatory  changes  should  be  made  that  would  diminish  the 
uniqueness  of  our  relationship  with  FFRDCs.  Since  this  recommendation  is  not 
Justified,  it  should  not  be  adopted. 
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